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Abstract 


This paper undertakes a review of the literature on writing cognition, writing instruction, and 
writing assessment with the goal of developing a framework and competency model for a new 
approach to writing assessment. The model developed is part of the Cognitively Based 
Assessments of, for, and as Learning (CBAL) initiative, an ongoing research project at ETS 
intended to develop a new form of kindergarten through Grade 12 (K-12) assessment that is 
based on modern cognitive understandings; built around integrated, foundational, constructed- 
response tasks that are equally useful for assessment and for instruction; and structured to allow 
multiple measurements over the course of the school year. The model that emerges from a 
review of the literature on writing places a strong emphasis on writing as an integrated, socially 
situated skill that cannot be assessed properly without taking into account the fact that most 
writing tasks involve management of a complex array of skills over the course of a writing 
project, including language and literacy skills, document-creation and document-management 
skills, and critical-thinking skills. As such, the model makes strong connections with emerging 
conceptions of reading and literacy, suggesting an assessment approach in which writing is 
viewed as calling upon a broader construct than is usually tested in assessments that focus on 
relatively simple, on-demand writing tasks. 

Key words: Formative assessment, writing instruction, literacy, critical thinking, reading, K-12, 
literature review, constructed-response 
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MODELING THE COGNITIVE BASIS FOR WRITING SKILL 


Introduction 

The purpose of this review is to examine the cognitive literature on writing, with an 
emphasis on the cognitive skills that underlie successful writing in an academic setting. ETS is 
attempting to design a writing assessment that will, under known constraints of time and cost, 
approximate full-construct representation. This is a long-term research project, not expected to 
produce an immediate product but intended to support innovation in both formative and 
summative assessment. Ultimately, successful innovation in writing assessment requires a 
synthesis of what is known about writing, both from cognitive and instructional perspectives, if 
writing assessment is to measure writing in terms that will be useful for teachers, students, and 
policymakers without doing violence to the underlying construct. 

Traditionally, academic writing instruction has focused on the so-called modes (e.g., 
expository, descriptive, narrative, and argumentative or persuasive; cf. Connors, 1981; Crowley, 
1998) and particularly on expository writing and argumentation. In this context, writing can be 
viewed relatively narrowly, as a particular kind of verbal production skill where text is 
manufactured to meet a discourse demand, or more broadly as a complex, integrated 
performance that cannot be understood apart from the social and cognitive purposes it serves. 
People write in order to achieve communicative goals in a social context, and this is as true of 
writing in a school context as anywhere else. Successful instruction will teach students the skills 
they need to produce a wide range of texts, for a variety of purposes, across a broad class of 
social contexts. This review will take that broader view, and as such we describe skilled writing 
as a complex cognitive activity, which involves solving problems and deploying strategies to 
achieve communicative goals. This review, therefore, while exploring the skills most relevant to 
each of the traditional modes, will argue for an approach to writing assessment that recognizes 
the importance of this larger context. 1 

Each of the various traditional modes and genres of academic writing deploys a different 
combination of skills, drawing variously from a wide range of reasoning skills, a variety of 
verbal and text production skills, and an accompanying set of social skills and schemas—all of 
which must be coordinated to produce a single end product that must stand or fall by the 
reception of its readership. Of the three most important traditional modes, ordinary narrative is 
often considered the easiest and is certainly the earliest taught. Exposition presupposes many of 
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the skills built into narrative texts and adds various strategies to support the communication of 
complex infonnation. Persuasive writing typically includes narrative and expository elements as 
needed, while adding persuasion and argumentation. Thus, an implicit hierarchy of proficiencies 
may be called upon in writing, varying considerably in cognitive complexity, ranging from 
simple narratives to complex forms of argument and the tour de force of literary skill. The 
traditional modes of writing are strictly speaking an academic construct, as skilled writers are 
able to produce texts that combine elements of all the traditional modes as required to achieve 
their immediate rhetorical purposes (cf. Rowe, 2008, p. 407). Traditional modes are of interest, 
however, insofar as they reflect genuine differences in the underlying skills needed to succeed as 
a writer. 

ETS is currently undertaking a long-term initiative, Cognitively Based Assessments of, 
for, and as Learning (CBAL), whose purpose is to develop a framework of cognitively grounded 
assessments focused on K-12 education. Such assessments will combine and link accountability 
assessments with formative assessments that can be deployed at the classroom level; the 
accountability assessments and fonnative assessments are intended to work together effectively 
to support learning. This review explicates the nature of writing skill as explored in the cognitive 
literature, with the immediate goal of identifying the elements of a proficiency model and the 
more distant goal of identifying what research needs to be done in order to meet the goals of the 
ETS CBAL initiative for writing. 


An Intersection of Fields 

This review of necessity has a wide scope and addresses literature in several disciplines: 
rhetoric and education, cognitive psychology, linguistics and computational linguistics, and 
assessment. Such an interdisciplinary focus is a direct consequence of the integrated nature of 
writing. A single piece of writing may do several things at once: tell a story; present facts and 
build a theory upon them; develop a logical argument and attempt to convince its audience to 
adopt a particular course of action; address multiple audiences; clarify the thinking of the author; 
create new ideas; synthesize other people’s ideas into a unique combination; and do it all 
seamlessly, with the social, cognitive, rhetorical, and linguistic material kept in perfect 
coordination. Consequently, the discussion that follows integrates materials from a variety of 
disciplines. 
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The challenge we face in this review is, first and foremost, one of identifying the skills 
that must be assessed if writing is to be measured in all its complexity, across a broad range of 
possible tasks and settings, but focused nonetheless on the kinds of writing skills that need to be 
taught in a K-12 school setting. In the early sections of this review, attention primarily is focused 
on the literature of writing cognition; later in the review, we shift attention to the literature of 
writing instruction and writing assessment and seek to outline a model of writing competence 
and a new approach to writing assessment designed to measure that model. 

Cognitive Models of Writing 

Cognitive models have tended to define writing in terms of problem-solving (cf. 
McCutchen, Teske, & Bankston, 2008). Generally, writing problems arise from the writer’s 
attempt to map language onto his or her own thoughts and feelings as well as the expectations of 
the reader. This endeavor highlights the complexity of writing, in that problems can range from 
strategic considerations (such as the organization of ideas) to the implementation of motor plans 
(such as finding the right keys on the keyboard). A skilled writer can confront a staggering 
hierarchy of problems, including how to generate and organize task-relevant ideas; phrase 
grammatically correct sentences that flow; use correct punctuation and spelling; and tailor ideas, 
tone, and wording to the desired audience, to name some of the more salient rhetorical and 
linguistic tasks. 

Clearly, writing skillfully can involve sophisticated problem solving. Bereiter and 
Scardamalia (1987) proposed that skilled writers often “problematize” a writing task, adopting a 
strategy they called knowledge transforming (pp. 5-6, 10-12, 13-25, 349-363). Expert writers 
often develop elaborate goals, particularly content and rhetorical goals, which require 
sophisticated problem-solving. In contrast, novice writers typically take a simpler, natural 
approach to composing, adopting a knowledge-telling approach in which content is generated 
through association, with one idea prompting the next (Bereiter & Scardamalia, pp. 5-30, 183- 
189, 339-363). Whereas the inefficient skills of novices may restrict them to a knowledge-telling 
approach, skilled writers can move freely between knowledge telling and knowledge 
transforming. 

Problem solving has been conceptualized in terms of information processing. In their 
original model, which has achieved broad acceptance in the field of writing research, Hayes and 
Flower (1980) attempted to classify the various activities that occur during writing and their 
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relationships to the task environment and to the internal knowledge state of the writer. Hayes and 
Flower posited that the writer’s long-tenn memory has various types of knowledge, including 
knowledge of the topic, knowledge of the audience, and stored writing plans (e.g., learned 
writing schemas). In the task environment, Hayes and Flower distinguished the writing 
assignment (including topic, audience, and motivational elements) from the text produced so far. 
Hayes and Flower identified four major writing processes: 

1. Planning takes the writing assignment and long-tenn memory as input, which then 
produces a conceptual plan for the document as output. Planning includes 
subactivities of generating (coming up with ideas), organizing (arranging those 
ideas logically in one’s head), and goal setting (detennining what effects one wants 
to achieve and modifying one’s generating and organizing activities to achieve local 
or global goals). 

2. Translating takes the conceptual plan for the document and produces text expressing 
the planned content. 

3. In reviewing, the text produced so far is read, with modifications to improve it 
(revise) or correct errors (proofread). 

4. Monitoring includes metacognitive processes that link and coordinate planning, 
translating, and reviewing. 

Hayes and Flower (1980) presented evidence that these processes are frequently 
interleaved in actual writing. For example, authors may be planning for the next section even as 
they produce already-planned text; they may read what they have written and detect how they 
have gone astray from one of their intended goals and then either interrupt themselves to revise 
the section they just wrote or change their goals and plans for the next section. In short, Hayes 
and Flowers concluded that writing involves complex problem solving, in which information is 
processed by a system of function-specific components. 

At this level of generality, Hayes and Flower’s (1980) framework is not particularly 
different from the kinds of schemes favored among rhetoricians, whether classical or modem. In 
the received Latin or Greek rhetorical tradition deriving from classical antiquity (cf. Corbett & 
Connors, 1999), for instance, the major elements are the following: 

• Invention (methods for coming up with ideas to be used in a text or speech), 
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• Arrangement (methods for organizing one’s content), 

• Style (methods for expressing one’s content effectively), 

• Memory (methods for remembering what one intends to say), and 

• Delivery (methods for actually presenting one’s content effectively). 

However, the emphasis that Hayes and Flowers (1980) put on these elements is rather 
different, since they intend their model to identify cognitive processes in writing, each of which 
presumably has its own internal structure and subprocesses that need to be specified in detail. In 
revising the original model, Hayes (1996) removed the external distinctions based upon task 
(e.g., the difference between initial draft and editing) in favor of an analysis that assumes three 
basic cognitive processes: (a) text interpretation, (b) reflection, and (c) text production. 

In this revised model, Hayes (1996) sought to identify how various aspects of human 
cognitive capacity interact with these tasks, distinguishing the roles of long-term memory, short¬ 
term memory, and motivation or affect. The Hayes (1996) model is specific about the contents of 
long-term memory, distinguishing among task schemas, topic knowledge, audience knowledge, 
linguistic knowledge, and genre knowledge. Similarly, Hayes (1996) specified how different 
aspects of working memory (e.g., phonological memory and visuospatial memory) are brought to 
bear in the cognitive processes of writing. While focusing on these cognitive dimensions, the 
model largely ignores distinctions at the task level. However, writing tasks differ in the types of 
problems they present to the writer, involving varying amounts of planning, translating, 
reviewing, or editing; thus, each task can call for a different combination of cognitive strategies. 

For our purposes, the distinctions among text interpretation, reflection, and text 
production are salient, in that they highlight three very different kinds of cognitive processes that 
are involved in almost any sort of writing task (i.e., the reflective, interpretive, and expressive 
processes). It is important, however, to note that, from a linguistic point of view, text production 
and text interpretation are not simple processes. When we speak about text production, for 
instance, it makes a great difference whether we are speaking about the realization of strategic, 
consciously controlled rhetorical plans or about automatized production processes, such as the 
expression of a sentence after the intended content is fully specified. Similarly, the process of 
text interpretation is very different, depending upon whether the object of interest is the 
phonological trace for the wording of a text, its literal interpretation, or the whole conceptual 
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complex it reliably evokes in a skilled reader. These varying levels of interpretation impose 
specific demands upon working and long-term memory. Consequently, we should distinguish 
between what Kintsch (1998) tenned the textbase (a mental representation of a text’s local 
structure) and the situation model (the fuller, knowledge-rich understanding that underlies 
planning and reviewing), especially when addressing writing at the highest levels of competence 
(including complex exposition and argumentation). 

If writing processes work together as a system, a question of primary importance is how 
content is retrieved from long-term memory. Writing effectively depends upon having flexible 
access to context-relevant information in order to produce and comprehend texts. In writing 
research, there has been considerable discussion about whether top-down or bottom-up theories 
better account for content generation (cf. Galbraith & Torrance, 1999). Early top-down theories 
of skilled writing (e.g., Bereiter & Scardamalia, 1987; Hayes & Flower, 1980) are based on the 
assumption that knowledge is stored via a semantic network, in which ideas are interconnected in 
various ways (Anderson, 1983; Collins & Loftus, 1975). In Hayes and Flower’s model, 
generating (a subcomponent of planning) is responsible for retrieving relevant information from 
long-term memory. Retrieval is automatic. Information about the topic or the audience serves as 
an initial memory probe, which is then elaborated, as each retrieved item serves as an additional 
probe in an associative chain. Similarly, Bereiter and Scardamalia described automatic activation 
as underlying a knowledge-telling approach. However, Bereiter and Scardamalia held that 
knowledge transformation depends upon strategic retrieval. In transforming knowledge, problem 
solving includes analysis of the rhetorical issues as well as topic and task issues, and that 
analysis results in multiple probes of long-term memory. Then, retrieved content is evaluated and 
selected, a priori, according to the writer’s goals (Alamargot & Chanquoy, 2001). Thus, 
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influential models of writing differ in their accounts of how retrieval happens in skilled writing. 

In proposing his knowledge-constituting model, Galbraith (1999) provided an alternative 
account of content retrieval, in which writing efficiency relies upon automatic activation. In 
contrasting knowledge constituting with knowledge transforming, he argued that complex 
problem solving alone cannot fully account for the experiences of professional writers. To 
describe their own writing experiences, professional writers often use the word discovery, since 
novel ideas often emerge spontaneously through the process of writing. Thus, planning occurs in 
a bottom-up fashion. The knowledge-constituting model provides a cognitive framework for 
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explaining this experience of discovery. In contrast to the semantic network (described above), 
Galbraith assumed that knowledge is stored implicitly, as subconceptual units within a 
distributed network (see Hinton, McClelland, & Rumelhart, 1990). Patterns of activation result 
from input constraints and the strength of fixed connections between nodes in the network. 
Accordingly, different ideas can emerge as a result of different patterns of global activation. 
Galbraith contended that competent writing involves a dual process, with one system rule based, 
controlled, and conscious (knowledge transforming) and the other associative, automatic, and 
unconscious (knowledge constituting). 

However conceptualized, all writing models hold that writing processes compete for 
limited cognitive resources. Writing has been compared to a switchboard operator juggling 
phone calls (Flower & Hayes, 1980) and an underpowered computer running too many programs 
(Torrance & Galbraith, 2005). The individual processes of planning, revising, and translating 
have shown to require significant cognitive effort (Piolat, Roussey, Olive, & Farioli, 1996). 
Working memory describes a limited-capacity system by which infonnation is temporarily 
maintained and manipulated (Baddeley, 1986; Baddeley & Hitch, 1974). Working-memory 
capacity has been linked closely to processes for reading, such as comprehension (Just & 
Carpenter, 1992; M. L. Turner & Engle, 1989), as well as to writing processes, such as 
translating fluency (McCutchen, Covill, Hoyne, & Mildes, 1994). 

Because of its limited capacity, writing requires managing the demands of working 
memory by developing automaticity and using strategies (McCutchen, 1996). With experience 
and instruction, certain critical, productive processes (e.g., those belonging to handwriting and 
text decoding) can become automatized and thus impose minimal cognitive load, freeing 
resources for other writing processes. Strategies (e.g., advance planning or postrevising) serve to 
focus attentional resources on a particular group of writing problems, improving the overall 
efficiency of problem solving. Knowledge telling represents an economical approach, which 
enables the writer to operate within the capacities of working memory; in contrast, knowledge 
transfonning is a costly approach that can lead to overloading working-memory resources. As 
writers become more competent, productive processes become increasingly automatic and 
problem solving becomes increasingly strategic. 
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Transcription A utomaticity 

In order for writing processes to function efficiently, transcription processes must become 
relatively automatized. The processes necessary for transcription vary across writing tools, as 
reflected in handwriting, typing, or dictating. Inefficient handwriting can slow text production 
while interfering with other writing processes (Bourdin & Fayol, 1994, 2000). Bourdin and Fayol 
(2000) found that working-memory load due to transcription interferes with word storage, a 
subprocess essential to text generation. By disrupting text generation via word storage, 
inefficient transcription may function like a bottleneck, allowing fewer language representations 
to get transformed into words on the page. 

Writing technology is not transparent. Although good writers may compose equally well with 
any writing tool (Gould 1980), the performance of poor writers can vary dramatically across tools. 
Handwriting, for example, is relatively complex, involving processes that include (a) retrieving 
orthographic representations from long-term memory, (b) parsing those representations into 
graphemes, (c) retrieving the forms for each grapheme, and (d) activating appropriate motor 
sequences. Compared to handwriting, typing (or word-processing) involves simpler graphemic 
processing and motor sequences and so may impose less transcription load on text generation, all 
else being equal. Bangert-Drowns (1993) conducted a meta-analysis of students composing via 
word-processing and found the effect sizes for studies of less skilled writers to be significantly 
higher than the effect sizes for studies of skilled writers. Speech-recognition technology leverages 
speech articulatory processes, which for most writers are relatively automated. Quinlan (2004) 
examined the effects of speech-recognition technology on composition by middle school students, 
with and without writing difficulties; he found that students with writing difficulties significantly 
benefited from speech-recognition technology by composing longer, more legible narratives. We 
have good reason to believe writing tools matter for children with writing difficulties. 

Reading Automaticity 

Reading plays a central role in competent writing (Hayes, 1996). Skilled writers often 
pause to reread their own texts (Kaufer, Hayes, & Flower, 1986), and such reading during 
writing has been linked to the quality of the written product (Breetvelt, van den Bergh, & 
Rijlaarsdam, 1996). During composing, reading can evoke other processes, such as planning (to 
cue retrieval of information from memory or to facilitate organizing), translating (to rehearse 
sentence wording), editing (to detect errors), or reviewing (to evaluate written text against one’s 
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goals). When composing from sources, writers may use reading strategies directed toward 
evaluating and selecting information in source documents. Not surprisingly, a writer’s ability to 
comprehend a source document determines his or her ability to integrate information from it. 
Revising also depends upon reading strategies. Reading is integral to knowledge transforming, 
since it provides an efficient means for defining and solving rhetorical problems. In terms of 
planning, reading the developing text may represent a flexible and powerful strategy for 
generating content by facilitating the activation of information in long-tenn memory (Breetvelt et 
ah; Hayes, 1996; Kaufer et ah). Given the potentially pervasive role of reading in writing, we can 
safely assume that most, if not all, writing-assessment tasks also measure some aspects of 
reading. 

Until reading processes become relatively automatic, they may interfere with or draw 
resources away from other writing processes. Dysfluent readers may be less able to critically 
read their own texts or adopt a knowledge-transforming approach; further, they may have 
difficulty integrating information from source texts. Consequently, in order for young writers to 
become competent writers, reading processes must become relatively automatic. 

Strategies to Manage the Writing Process 

In addition to automaticity, writing well depends upon using strategies. At any given 
moment, a writer potentially faces a myriad of hierarchically interrelated problems, such that one 
change can affect other things. Given that writers can cope with relatively few problems during 
drafting, strategies afford a systematic means for approaching these problems. All writing 
strategies work by focusing attentional resources on a specific group of writing problems, which 
generally relate to either planning or evaluating. Strategic approaches may be broadly grouped 
into top-down and bottom-up approaches. The top-down approach is characterized by advance¬ 
planning strategies, such as outlining and concept maps. By frontloading some idea generation 
and organization, thereby resolving macrostructural text issues early in the writing session, the 
writer may find drafting easier and more effective. In contrast, the bottom-up approach assumes 
that writers discover new and important ideas as their words hit the page. The bottom-up 
approach is characterized by much freewriting and extensive revising, as advocated by Elbow 
(1973, 1981). In other words, the act of composing can prompt new ideas, which might not 
otherwise emerge. Also, a bottom-up approach, which features extensive freewriting, may be an 
effective exercise for helping improve handwriting or typing fluency (see Automaticity section 


9 



below; also see Hayes, 2006). The top-down approach enjoys more empirical support than the 
bottom-up approach. That is, numerous studies have found that making an outline tends to lead 
to the production of better quality texts. However, both have a sound theoretical basis, in that 
both approaches isolate idea generating or organizing from drafting. Each approach has its own 
more or less loyal following among language arts teachers. 

Planning 

Much planning happens at the point of inscription, as writers pause to think about what 
they will write next (Matsuhashi, 1981; Schilperoord, 2002). This real-time planning requires 
juggling content generation and organization with other writing processes, such as text 
generation and transcription. Consequently, real-time planning can place a considerable load 
upon working memory. As a strategy, advance planning can reduce working-memory demands 
by frontloading and isolating some planning-related activities, thus simplifying things at the 
point of inscription. 

Younger, typically developing children tend to do little advance planning (Berninger, 
Whitaker, Feng, Swanson, & Abbott, 1996), and children with learning disabilities typically plan 
less than developing children (Graham, 1990; MacArthur & Graham, 1987). Moreover, writers 
who use advance planning strategies tend to produce better quality texts (Bereiter & 

Scardamalia, 1987; De La Paz & Graham, 1997a, 1997b; Kellogg, 1988; Quinlan, 2004). In a 
study of undergraduates who were writing letters, Kellogg found that a making an outline 
improved letter quality, whether the outline was handwritten or constructed mentally. In the 
outline condition, undergraduates devoted a greater percentage of composing time to lexical 
selection and sentence construction (i.e., text generation), relative to planning and reviewing. 
Moreover, participants in the outlining condition spent significantly more time composing their 
letters. Kellogg concluded that outlining facilitated text quality by enabling writers to 
concentrate more upon translating ideas into text (i.e., text generation). Quinlan found similar 
results in his study of middle school children who were composing narratives. The results of 
these studies suggest that advance-planning strategies improve overall writing efficiency. 
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Revising 

Competent writers often revise their texts (Bereiter & Scardamalia, 1987). Hayes and 
Flower (1980) distinguished between editing—the identification and correction of errors (more 
properly tenned copyediting or proofreading)—and revising, in which the writer aims to improve 
the text. Together, editing and revising encompass a wide range of writing problems. For 
example, detecting various types of typographical errors can involve processing various types of 
linguistic information, including orthographic, phonological, syntactic, and semantic (Levy, 
Newell, Snyder, & Timmins, 1986). In the revising model proposed by Hayes, Flower, Schriver, 
Stratman, and Carey (1987), revising involves comprehending, evaluating, and defining 
problems. Hayes (2004) described revising as largely a function of reading comprehension. In 
their study of children’s revising, McCutchen, Francis, and Kerr (1997) concluded that writers 
must become critical readers of their own texts in order to assess the potential difficulties their 
readers might encounter. 

Like planning, revising can happen at any time. Postdraft revising should be considered a 
strategy that serves to isolate evaluative problems and focus analysis at pertinent levels of the 
text. Identifying errors in word choice or punctuation demands a close reading of words and 
sentences that goes beyond basic comprehension processes. However, skilled revising that leads 
to meaning-level changes requires additional reading strategies. Palinscar and Brown (1984) 
found that experienced readers employed six strategies in the course of comprehending a text, all 
of which may transfer to revising: (a) Understand the implicit and explicit purposes of reading, 
(b) activate relevant background knowledge, (c) allocate attention to major content, (d) evaluate 
content for internal consistency, (e) monitor ongoing comprehension, and (f) draw and test 
inferences. Reading strategies for comprehending overlap with reading strategies for revising. 
McCutchen et al. (1997) found that high- and low-ability students employed different reading 
strategies when asked to revise texts. High-ability students described using a skim-through 
strategy that included rereading the entire text after surface-level errors had been found. In 
contrast, lower ability writers often used a sentence-by-sentence reading strategy that was not 
effective in diagnosing meaning-level problems. 

To summarize, we can describe skilled writing as a complex cognitive activity that 
involves solving problems and deploying strategies to achieve communicative goals. According 
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to existing models of writing competency (Bereiter & Scardamalia, 1987; Hayes, 1996; Hayes & 
Flower, 1980), writers typically encounter three challenges: 

1. Planning a text (including invention) involves reflective processes in which the 
author builds up a situation model including his or her own goals, the audience and 
its attitudes, and the content to be communicated, and develops a high-level plan 
indicating what is to be communicated and how it is to be organized. In some cases, 
this plan may correspond closely to the textbase-level content of the final document, 
although in real writing tasks the original plan may leave much unspecified, to be 
fleshed out iteratively as the drafting process proceeds. 

2. Drafting a text (text production) is the expressive process by which the intended 
document content, at the textbase level, is converted into actual text. This process 
includes planning at the rhetorical level as well as more automated processes of 
converting rhetorical plans into text. 

3. Reading a text (or text interpretation, one aspect of reviewing in the Hayes & 
Flower, 1980, model) is the interpretive process by which the author reads the text 
he or she has produced and recovers the textbase-level information literally 
communicated by the text. In so doing, the author may analyze the text at various 
levels, including orthographic, syntactic, and semantic, for purposes of reflection, 
evaluation, or error detection. 

Successfully addressing these challenges to produce a satisfactory text requires the 
coordination of multiple processes that draw heavily upon limited cognitive resources. 

Efficiently solving problems—while avoiding cognitive overload—requires the development of 
automaticity in productive processes and strategy use in executively controlled processes. 

Writing as Social Cognition 

The literature we have reviewed thus far focused on writing entirely within a cognitive 
psychology perspective, in which the focus is entirely on what happens within the writer’s head. 
Another perspective on writing takes into account the fact that the cognitive skills that writers 
deploy are socially situated and take place in social contexts that encourage and support 
particular types of thinking. Sociocultural approaches to writing (for an extensive review, see 
Prior, 2006) emphasize that writing is (a) situated in actual contexts of use; (b) improvised, not 
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produced strictly in accord with abstract templates; (c) mediated by social conventions and 
practices; and (d) acquired as part of being socialized into particular communities of practice. 

The sociocultural approach emphasizes that the actual community practices deeply 
influence what sort of writing tasks will be undertaken, how they will be structured, and how 
they will be received, and that such constructs as genres or modes of writing are in fact 
conventional structures that emerge in specific social contexts and exist embedded within an 
entire complex of customs and expectations. Thus, Heath (1983) showed that literate practices 
vary across classes within the same society and that the cultural practices of home and 
community can reinforce, or conflict with, the literacy skills and expectations about writing 
enforced in school. Various sociocultural studies (Bazennan, 1988; Bazennan & Prior, 2005; 
Kamberelis, 1999; Miller, 1984) have shown that genres develop historically in ways that reflect 
specific changes and development in community structure and practice. In short, the purposes for 
which writing is undertaken, the social expectations that govern those purposes, the specific 
discourse forms available to the writer, the writing tools and other community practices that 
inform their practice—all of these reflect a larger social context that informs, motivates, and 
ultimately constitutes the activities undertaken by a writer. Writing skills subsist in a social space 
defined by such contexts and the institutions and practices associated with them. 

The kind of writing with which we are concerned in this review is, of course, school 
writing: the kind of writing that is socially privileged in an academic context and which is a 
critical tool for success in a variety of contexts, including large parts of the academic and 
business worlds of 21 st -century Western society. We make no particular apology for this 
limitation, as it is directly driven by our ultimate purpose—supporting more effective assessment 
and instruction in writing in a school context—but it is important to keep this limitation of scope 
in mind. Many of the cognitive demands of this sort of writing are driven by the need to 
communicate within a literate discourse community where interactions are asynchronous; are 
mediated by publication or other methods of impersonal dissemination; and often involve 
communication about content and ideas where it is not safe to assume equal knowledge, high 
levels of interest or involvement, or sharing of views. 

Some of the most interesting work examining the social context of school writing can be 
found in the later work of Linda Flower (Flower, 1990, 1994; L. D. Higgins, Flower, & Long, 
2000). For instance, Flower (1990) studied the transition students undergo as college freshmen, 
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when they must learn how to read texts in ways that enable them to write effectively according to 
the social expectations of the university context. She explored how the expectations of that social 
context clash with the practices and assumptions students bring from their writing experiences in 
a secondary school context. L. D. Higgins et al. explored in depth the social practices and related 
cognitive skills required to successfully conduct the kinds of inquiry needed to write well in 
social contexts where they are required to generate, consider, and evaluate rival hypotheses. 

While much of the rest of this review focuses on a fairly fine-detail analysis of specific 
cognitive skills and abilities needed to perform well on characteristic types of academic writing, 
it is important to keep in mind that all of these exist as practices within a particular discourse 
setting and cultural context. To the extent that both writing instruction and writing assessment 
are themselves cultural practices, and part of this same context, it is important to keep in mind 
that cultural communities provide the ultimate measure of writing effectiveness. Assessment 
should be focused on whether students have acquired the skills and competencies they need to 
participate fully in the discourse of the communities that provide occasions for them to exercise 
writing skills. 

In particular, it is important to keep in mind the many specific cognitive processes that 
are involved in writing while not losing sight of the fact that they are embedded in a larger social 
situation. That situation can be quite complex, involving both an audience (and other social 
participants, such as reviewers, editors, and the like) and a rich social context of well-established 
writing practices and a variety of social conventions and institutions. Figure 1 may be useful as a 
way of conceptualizing how the purely cognitive processes of writing are situated within a larger 
social context. One of the tensions that results—to be discussed in the final sections of this 
document—involves the conflict between a need to assess writing globally (being sensitive to 
social context and rhetorical purpose) and the need to measure a variety of specific skills and 
abilities that form important components of expert writing. 

The various elements mentioned in Figure 1 are not equally important in all writing tasks. 
One of the complexities of writing, viewed as a skill to be taught, learned, or assessed, is that 
there are so many occasions for writing and thus so many different specific combinations and 
constellations of skills and abilities that may be required for writing success. We may note, by 
way of illustration, that writing a personal letter draws on different skills than writing a research 
paper and that yet another constellation of skills may be required to write a successful letter to 
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Figure 1. Essential dimensions of writing proficiency. LTM = long-term memory. 








the editor. To create a general model of writing, it is important to have a picture of what skills 
may be called upon in particular writing situations and to have a model that indicates how 
different writing occasions will draw differentially upon these skills. These kinds of 
considerations are the focus of the next section of the paper. We discuss the classic modes and 
genres of writing—especially the narrative, expository, and persuasive modes. We do not believe 
that these modes are privileged in the sense that traditional writing pedagogies assume, but rather 
that they can provide a useful glimpse into how writing can vary in cognitively interesting ways. 
To a significant extent the so-called modes of writing illustrate very different rhetorical purposes 
and very different combinations of skills to support these purposes. Insofar as expert writers 
must be able to succeed as writers in a variety of specific occasions, for a variety of purposes, 
and across a wide range of social contexts, a detailed picture of the kinds of skills demanded of 
writers across the traditional modes is useful. 

Differential Recruitment of Cognitive Abilities in Writing 

Each genre—indeed, every occasion for writing—presents specific, problem-solving 
challenges to the writer. By design, cognitive models of writing do not directly address the 
specific problems inherent to each genre (much less to individual writing tasks). There is, 
however, strong evidence in the literature that the cognitive demands of writing tasks vary 
significantly and therefore should not be treated as essentially unifonn. In particular, there is 
some evidence that competence in everyday narrative is acquired relatively early and 
competence in persuasive writing relatively late, and that the same gradient of competence 
applies across grade levels (cf. Applebee et al., 1990; Bereiter & Scardamalia, 1987; Britton et 
ah, 1975; Greenwald et al., 1999; Knudson, 1992). The implication is that the specifics of the 
writing task make a very large difference in performance, and that these differences are 
particularly problematic for novice writers. We expect that composing in different genres 
(narration, exposition, and argumentation) places varying demands upon the cognitive system. 
The traditional categories of expository, persuasive, and narrative and descriptive writing should 
not be viewed in the discussion that follows as of interest in their own right. That is, we are not 
treating them as absolutes or as unanalyzed categories, but instead as prototypical modes of 
thought, exemplars of historically identifiable discourse practices that are important within the 
context of school writing. By comparing the cognitive requirements of each, we can obtain a 
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much richer picture of variations in the requirements of writing and thus a more robust cognitive 
model of writing viewed more generally. 

One theme we wish to explore in particular involves ways in which the most distinctively 
academic sort of school writing—persuasive, argumentative writing—differs from narration and 
exposition. Many of the differences have an obvious impact on the component tasks of writing. 
Perhaps the four most important are (a) methods of text organization and their relationship to 
domain knowledge and working memory, (b) the role of the audience, (c) mastery of textual 
cuing skills and other writing schemas appropriate to specific modes of writing, and (d) the role 
of reasoning skills. Since the goal of writing instruction is to inculcate competence across the 
board and in a variety of genres, and indeed in a variety of specific purposes and situations, it is 
important to consider some of the factors that may account for why certain kinds of writing 
(including many forms of expository and persuasive writing) may present greater challenges to 
the writer. 


The Role of the Audience 

The matter of audience poses a central problem to the writer. The difference between 
writing and dialogue is precisely the absence of the audience during the composition process and 
hence the need for the writer to mentally simulate the audience’s reactions and responses and, if 
necessary, to account for those responses in the produced text. According to one developmental 
hypothesis, young children may be able to produce text corresponding to a single turn in a 
conversation (Bereiter & Scardamalia, 1987). There is clear evidence that when students are 
provided with a live audience—such as another student responding to their writing or 
collaborating with them in producing the text—the quality and length of children’s writing 
output increase (e.g., Calkins, 1986; Daiute, 1986; Daiute & Dalton, 1993). In the case of 
argumentation in particular, Kuhn, Shaw, and Felton (1997) demonstrated that providing 
students with a partner for live interaction increases the quality of the resulting text. The essence 
of argumentation is that the author must write for people who do not necessarily share 
background assumptions or perspective; thus, the task of constructing an argument requires not 
only that the writer come up with reasons and evidence, but also that those reasons and evidence 
be selected and presented to take the viewpoints, objections, and prejudices of the audience into 
account. In effect, whatever skills students may possess in dialogic argumentation must be 
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redeployed in a fundamentally different, monologic setting, which presents its own challenges 
along the lines discussed in Reed and Long (1997). 4 

This is not to say that attention to the audience is irrelevant to narration or exposition: 
quite the opposite. Flower (1979) noted, for instance, that one of the differences between 
experienced writers and college freshmen was that the expository writing of freshmen tended to 
be topic bound, whereas experienced writers made significant adjustments of presentation based 
upon audience considerations. McCutchen (1986) noted that children’s planning processes for 
writing typically focus on content generation rather than on the development of the more 
sophisticated (and often audience-sensitive) types of plans developed by more experienced 
writers. One issue that has been raised in the literature, but does not appear to have been 
resolved, is the relationship between development of writing skills and audience sensitivity 
across the various modes of writing. Some would argue (cf. Eliot, 1995) that narrative, precisely 
because it is more deeply embedded in most novice writers’ experiences and social worlds, is 
less alienating and thus provides a more natural bridge to developing writing skill in all modes of 
writing. 

Part of an author’s sensitivity to audience depends on sensitivity to how an audience 
processes different types of texts. In persuasive text, the primary issue is the audience’s 
willingness to believe the writer’s arguments. Thus, the critical audience-related skill is the 
writer’s capacity to assess what kinds of arguments and what sorts and quantities of evidence 
should be marshaled to establish a point. In expository text, on the other hand, one of the key 
issues is to determine what has to be said explicitly and what can be left implicit, which may 
depend partly on the audience’s degree of background knowledge. McNamara, Kintsch, Songer, 
and Kintsch (1996) have shown that high-cohesion texts facilitate comprehension for readers 
with low domain knowledge, but that readers with high domain knowledge learn more from low- 
cohesion expository texts, because they actively infer the necessary connections and thus build a 
more complete mental model of text content. This result helps explain why an author writing 
expository text for experts will make fundamentally different choices regarding details to spell 
out in full than will an author writing for novices. A related issue is the situational interest an 
author of expository writing can expect to evoke in the audience, which is partly a matter of style 
but also appears to intersect strongly with prior topical knowledge (Alexander, Kulikowich, & 
Schulze, 1994). 
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Children’s abilities in these areas also emerge first in conversational settings, but it would 
be a mistake to assume a simple, black-and-white picture. See, for example, Little (1998), who 
indicated that 5- to 9-year-olds can make adjustments in an expository task for an absent 
audience, but that their performance with a live, interacting audience is significantly richer. 

The implication of these considerations for a proficiency model is that we must 
distinguish along two dimensions: awareness of the audience and ability to adjust content and 
presentation to suit an audience. Awareness of the audience ranges from immediate knowledge 
of a present audience through intermediate degrees to abstract knowledge of the expectations of 
an entirely hypothetical, absent audience. Adjustment to content covers a wide range. One kind 
of adjustment is typical for exposition and ties content and stylistic choices to the audience’s 
knowledge state and degree of interest in the subject. Another kind of adjustment is typical in 
persuasive writing and focuses on the selection of arguments or rhetorical strategies to maximize 
persuasive impact. This kind of sensitivity to the audience is one of the hallmarks of a 
sophisticated writer. 

Organization, Domain Knowledge, and Working Memory 
Writers face the problem of how to organize their texts, a decision that to a great extent 
may be dictated by genre. The organization of narration and, to a lesser extent, of exposition 
derives from structures intrinsic to the content domain. That is, the structure of the narrative is 
fairly strongly determined by the content of the story to be told. Similarly, if one wishes to 
present factual material, some degree of natural conceptual organization is intrinsic to the 
material to be presented. For instance, to describe the cell, it would be natural to organize the 
presentation in terms of the parts of the cell and their function. 

By its nature, argumentation is more abstract and deploys common patterns of reasoning 
to organize potentially quite disparate materials. The implication is that domain knowledge of the 
sort that children are likely to have, or to acquire from content instruction, may provide 
significant inferential support both in the planning stage (when the writer must decide how to 
structure the text) and in reading (when the reviewer or reader must decide how the material is in 
fact organized). The fact that topical familiarity is known to increase writing quality (e.g., 
DeGroff, 1987; Langer, 1985; McCutchen, 1986) thus may be explained partially, at least in the 
cases of narrative (cf. Hidi & Hildyard, 1983) and, to a lesser extent, exposition. Conversely, the 
relative importance of rhetorical relations rather than topical relations suggests that topical 
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information may not have as strong a facilitating effect on argumentation, though there is mixed 
evidence on this question (Andriessen, Coirier, Roos, Passerault, & Bert-Erboul, 1996; De 
Bernardi & Antolini, 1996). 

The connection between domain knowledge (a long-term memory resource) and working 
memory may play an important role in the writing process, particularly in expository writing or 
in portions of other text types with an expository structure. Text planning and production are 
intensively memory-based processes that place high demands on working memory; that is, 
writing performance depends critically upon being able to recall relevant knowledge and 
manipulate it in working memory. For instance, Bereiter and Scardamalia (1987) showed a 
correlation between writing quality and performance on tasks measuring working-memory 
capacity. Hambrick (2001) reported that recall in a reading-comprehension task is facilitated 
both by high working-memory capacity and high levels of domain knowledge. Similar 
interactions appear to take place in text planning and generation, where it can be argued that 
long-term knowledge activated by concepts in working memory functions in effect as long-term 
working memory, expanding the set of readily accessible concepts. However, the supposition 
that domain knowledge is likely to have particularly facilitating effects on the writing of 
expository texts (as opposed to narrative or persuasive texts) has not been studied extensively, 
and caution should be exercised in this regard. Argumentation, when well executed, typically 
presupposes exposition as a subgoal, so that disentangling differential effects may prove 
difficult. 

The implication for a proficiency model is that topic-relevant prior knowledge and 
working-memory span are both relevant variables that strongly can affect writing quality and 
therefore must be included in a cognitive model, though their relative impact on writing quality 
may vary across genres, with background knowledge perhaps having a stronger overall effect on 
expository than on persuasive writing quality. To the extent that preexisting background 
knowledge facilitates writing, we must concern ourselves with the differential impact that the 
choice of topic can have upon writers. Given the high levels of cognitive demand that skillful 
writing involves, writers who already have well-organized knowledge of a domain and 
concomitant interest in it may have significant advantages and be able to demonstrate their 
writing abilities more easily. 
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Mastery of Textual Cues and Other Genre Conventions 

Competent writers also must master the use of textual cues, which vary by genres. The 
discourse markers specifically used to indicate narrative structure cover several conceptual 
domains, including time reference, event relationships, and perspective or point of view, as well 
as such grammatical categories as verb tense, verbs of saying, and discourse connectives, among 
others (Halliday & Hasan, 1976). These kinds of devices appear to be acquired relatively early 
and to be strongly supported in oral discourse (Berman, Slobin, Stromqvist, & Verhoeven, 1994; 
Norrick, 2001), though many features of literary narrative may have to be learned in the early 
school years. 

Mastery of the textual cues that signal expository organization also appears to be 
relatively late in developing, and knowledge of expository, as opposed to narrative, writing 
appears to develop more slowly (Englert, Stewart, & Hiebert, 1988; Langer, 1985). The use of 
textual cues in writing is, of course, likely to be secondary to their more passive use in reading, 
and there is evidence that skilled readers, who evince an ability to interpret textual cues 
effectively, are also better writers (Cox, Shanahan, & Sulzby, 1990). 

In argumentation in text, discourse markers signal the relationship among the parts of a 
text (cf. Azar 1999; Mann & Thompson 1988), and there is good reason to think that the ability 
to interpret the relevant linguistic cues is not a given and must be developed during schooling. 

For instance, Chambliss and Murphy (2002) examined the extent to which fourth- and fifth-grade 
children are able to recover the global argument structure of a short text, and these researchers 
found a variety of levels of ability, from children who reduce their mental representation of the 
text to a simple list, up to a few who show evidence of appreciating the full abstract structure of 
the argument. Moreover, many of the linguistic elements deployed to signal discourse structure 
are not fully mastered until relatively late (Akiguet & Piolat 1996; Donaldson, 1986; Golder & 
Coirier, 1994). 

Knowing what cues to attach to an argument is a planning task, whereas recovering the 
intended argument, given the cue, is an evaluative task. Given the considerations we have 
adduced so far—that argumentation entails attention to audience reaction and uses cues and 
patterns not necessarily mastered early in the process of learning to write—a strong connection 
with the ability to read argumentative text critically is likely. That is, persuasive text, to be well 
written likely requires that the author be able to read his or her own writing from the point of 


21 



view of a critical reader and to infer where such a reader will raise objections or find other 
weaknesses or problems in the argumentation. This kind of critical reading also appears to be a 
skill that develops late and that cannot be assumed to be acquired easily (Larson, Britt, & Larson, 
2004). 

The key point to note here is that particular genres of writing use specific textual cues as 
the usual way of signaling how the concepts cohere; whereas the ability to deploy such cues 
effectively is in part a translational process (to be discussed below), a prior skill—the basic 
comprehension of the devices and techniques of a genre—also needs to be learned, which 
precedes the acquisition of the specific skills needed to produce text meeting genre expectations. 
This kind of general familiarity is necessary for verbal comprehension and is clearly important as 
a component skill in assessing audience reaction. The implication for a proficiency model is that 
the level of exposure to persuasive and expository writing (and the degree to which students have 
developed reading proficiency with respect to those types of text) is likely to provide a relevant 
variable. We cannot assume that writers will have mastered the characteristic linguistic and 
textual organization patterns of particular genres or types of writing without having significant 
exposure to them both in reading and in writing. 

Critical Thinking and Reasoning 

Educators often have contended that clear writing makes for clear thinking, just as clear 
thinking makes for clear writing. It is, of course, possible to write text nearly automatically with 
a minimum of thought, using preexisting knowledge and a variety of heuristic strategies, such as 
adopting a knowledge-telling approach. For many writing tasks, such an approach may be 
adequate. However, sophisticated writing tasks often pose complex problems that require critical 
thinking to solve (i.e., knowledge transforming). Thus, from an educational perspective, students 
should be able to use writing as a vehicle for critical thinking. Accordingly, writing should be 
assessed in ways that encourage teachers to integrate critical thinking with writing. 

Indeed, many of the skills involved in writing are essentially critical-thinking skills that 
are also necessary for a variety of simpler academic tasks, including many kinds of expository 
writing. Constructing an effective expository text presupposes a number of reasoning skills, such 
as the ability to generalize over examples (or relate examples to generalizations), to compare and 
contrast ideas, to recognize sequences of cause-effect relationships, to recognize when one idea 
is part of a larger whole, to estimate which of various ideas is most central and important, and so 
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on. These skills in combination comprise the ability to formulate a theory. Understanding an 
expository text is in large part a matter of recovering such abstract, global relationships among 
specific, locally stated facts. Writing expository text presupposes that the reader is capable of 
recognizing and describing such relationships and, at higher levels of competency, of 
synthesizing them from disconnected sources and facts. 

There is clear evidence that, at least at the level of text comprehension, children are much 
less effective at identifying such relationships than they are at identifying the relatively fixed and 
concrete relationships found in narratives. In particular, children tend to be much more effective 
at identifying the integrating conceptual relationships in narrative than in expository text and 
tend to process expository text more locally in terms of individual statements and facts (Einstein, 
McDaniel, Bowers, & Stevens, 1984; McDaniel, Einstein, Dunay, & Cobb, 1986; Romero, Paris, 
& Brem, 2005). Compared to narrative, manipulation of expository text to reduce the explicit 
cueing of global structure has a disproportionately large negative impact on comprehension, and 
manipulating expository text to make the global structure explicit has a disproportionately large 
positive impact (McNamara et ah, 1996; Narvaez, van den Broek, & Ruiz, 1999). Conversely, 
manipulations of text to increase text difficulty can improve comprehension among poorer 
readers, but only if they force the poorer readers to encode relational information linking the 
disparate facts presented in an expository text (McDaniel, Hines, & Guynn, 2002). These results 
suggest that recognition of abstract conceptual relationships is easy when cued explicitly in a 
text, hard when not, and absolutely critical for comprehension. 

The implications for writing are clear. Students having trouble inferring conceptual 
relationships when reading an expository text are likely also to have trouble inferring those 
relationships for themselves when asked to conceptualize and write an expository text. 

Beginning writers of expository text thus may have greater difficulties than they would with 
narrative because they lack the necessary conceptual resources to structure their knowledge and 
make appropriate inferences; on the other hand, they are likely to have fewer difficulties with 
argument, where the nature of the task intrinsically requires dialogic thinking even in situations 
where no audience response is possible. As with argumentation, the skills involved in exposition 
appear to follow a clear developmental course. Inductive generalization (generalization from 
examples to a category) and analogy (generalization from relationships among related concepts) 
tend to be driven strongly by perceptual cues at earlier ages (up to around the age of 5) before 
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transitioning to more abstract, verbally based patterns by the age of 11 (Ratterman & Gentner, 
1998; Sloutzky, Lo, & Fisher, 2001). 

The importance of reasoning skill is even more evident in the case of persuasive writing 
tasks. Kuhn (1991) reported that about half of her subjects in an extensive study failed to display 
competence in the major skills of informal argument, a result well supported in the literature 
(Means & Voss, 1996; Perkins, 1985; Perkins, Allen, & Hafiner, 1983). Although children 
display informal reasoning skills at some level at an early age and continue to develop in this 
area, relatively few become highly proficient (Felton & Kuhn, 2001; Golder & Coirier, 1996; 
Pascarelli & Terenzini, 1991; Stein & Miller, 1993). However, there is evidence that students are 
capable of significant performance gains when instructed in reasoning skills, at least when it is 
explicit and involves deliberate practice (Kuhn & Udell, 2003; van Gelder, Bissett, & Gumming, 
2004). One concern is that argumentation skills may involve underlying capabilities that mature 
relatively late and thus involve metacognitive and metalinguistic awareness (Kuhn, Katz, & 
Dean, 2004). 

The development of children’s argumentation skills appears to begin with interpersonal 
argumentation with a familiar addressee (R. A. Clark & Delia, 1976; Eisenberg & Garvey, 

1981). Expression of more complex argumentation skills tends to increase with age and is most 
favored when children are familiar with the topic and situation, personally involved, and easily 
can access or remember the data needed to frame the argument (Stein & Miller, 1993). 
Conversely, the greatest difficulties in producing effective argumentation appear to be connected 
with the need to model the beliefs and assumptions of people significantly different than oneself 
(Stein & Bernas, 1999). 

One implication is that it is not safe to assume that writers have mastered all of the 
reasoning skills presupposed for effective writing. Another is that it is probably wise not to treat 
writing skill as somehow separate from reasoning; argumentation and critical thinking are 
especially interdependent. 


Planning and Rhetorical Structure 

The literature reviewed thus far has suggested differences in the cognitive demands made 
by each of the three traditional modes (and, by implication, for other, less traditional task types). 
These differences appear to have implications for every cognitive activity involved in writing. In 
the case of argumentation, for instance, planning is relatively complex, because the schemas and 
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strategies needed for effective argument are less common; revision is harder to do effectively, 
because it presupposes the ability to read critically, to raise objections to arguments, and to 
imagine how someone other than oneself would respond to one’s own work product. Without the 
expectations for good argument, the skills required for effective exposition remain, and similar 
arguments apply. Even if the intended concepts have been fully developed in a writer’s mind, 
however, the translation process also likely involves significant complexities. One cannot simply 
take for granted the ability to take a writing plan and convert it into an appropriate text. 

In the case of persuasive writing, these complexities can be viewed at two levels. An 
argument is by its nature a complex structure that can be presented in many alternative orders: 
Claims, evidence, warrants, grounds, and rebuttals can be nested and developed in parallel, so 
that the problem of taking an argument and deciding on the proper order to present the material 
is not trivial. Thus, the task of creating a document outline—whether explicitly or implicitly—is 
an area where argumentative writing is probably more complex than many other forms of 
writing. Even when the document structure is well planned, this structure must be signaled 
appropriately. Language is amply endowed with devices for expressing the structure and 
relationships among the pieces of a text, including its argumentative structure. A key skill that 
writers must develop is the ability to structure their texts in such a way that these devices 
unambiguously cue that structure. 

Similar points can be raised with respect to expository writing. Here again, there is a 
multidimensional structure, a network of related ideas that have to be linearized and then turned 
into a stream of words, phrases, clauses, and sentences. Even with an excellent plan for 
presenting the material in linear order, the translation process need not be straightforward, and 
the quality of the resulting text depends crucially on how the author chooses to move from plan 
to written text. 

Narrative is not exempt from these complexities; it too can pose issues in text production, 
since the author must decide what information to present (which depends in part on viewpoint 
and perspective) and what to deemphasize or eliminate. There is much more to the structure of a 
narrative than a simple event sequence (cf. Abbott, 2002; D. Hennan, 2003). 

The process of producing text, regardless of genre, involves a complex mapping to what 
is explicitly present in the text from the actual content that the author intends to present. We may 
distinguish at least three kinds of linguistic knowledge that fonn this part of the process: (a) 
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document structure templates and other types of document plans; (b) the general linguistic 
indications of rhetorical structure, which are typically used to signal the boundaries and 
transitions among the elements of particular document structure templates; and (c) more general 
devices and resources for linguistic cohesion. Few of these elements are specific to any one 
genre of writing, but all of them need to be mastered to achieve effective argumentation. In 
effect, they fonn part of the complex of cues and expectations that define genres in the minds of 
the reader, and, as such, these elements comprise a learned, social category whose mastery 
cannot be taken for granted. 

Document-Structure Plans, Rhetorical Structure, and Discourse Cues 

Obviously, many typical organizational patterns are more or less obligatory for particular 
writing tasks. Such “chunked” organizational schemes are known to facilitate effective writing. 
The classic school essay, with its introduction, three supporting paragraphs, and conclusion, is 
one such template. A wide variety of document templates exists, typically in genre-specific 
forms. For instance, Teufel and Moens (2002) presented a template for structuring scientific 
articles that depends critically upon the fact that scientific articles involve a well-defined series 
of rhetorical moves, each typically occupying its own block or section and strongly signaled by 
formulaic language, rhetorical devices, and other indications that are repeated across many 
different essays from the same genre. Formulaic document-structure templates of this sort are 
intrinsic to the concept of genre; one textual genre differs from another to a large extent by the 
kinds of formulaic organizational patterns each countenances. Note, however, the potential 
disconnect between stored writing plans of this type and the presence of a valid argument. It is 
entirely possible to structure a document to conform entirely to the template for a particular type 
of writing (e.g., a scientific paper) and yet for the content not to present a valid argument. 
However, that situation may reflect partial learning, in which a template has been memorized 
without full comprehension of the pattern of reasoning that it is intended to instantiate. 5 

Document-structure plans can occur at varying levels of complexity. In the case of 
argumentation, for instance, a common document-organization principle involves grouping 
arguments for and against a claim. Bromberg and Doma (1985) noted that essays involving 
multiple arguments on both sides of an issue typically are organized in one of three ways: (a) a 
block of proarguments, followed by a block of antiarguments; (b) a sequence of paired pro- and 
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antiarguments; and (c) in-depth development of proarguments, with antiarguments integrated 
into and subordinated to the main line of reasoning. 

As the granularity of analysis moves from the whole document to the level of individual 
paragraphs and sentences, the appropriate unit of analysis shifts from fairly fixed templates to 
rhetorical moves the author wishes to make and the corresponding ways of expressing those 
rhetorical moves in text units. At this level, theories of discourse such as rhetorical structure 
theory (RST; Mann & Thompson, 1987) provide methods for detailed analysis of text structure. 
Many of the relationships postulated in RST correspond to structures of argumentation, 
involving relationships such as evidence, elaboration, motivation, concession, condition, reason, 
or justification. However, RST is explicitly a theory of the structure of the text, and its 
relationships are defined as relationships among text units, through relationships that correspond 
to the intended rhetorical moves. 

The critical point to note about RST is that it is a theory of the encoding, or translation 
process, specifying how particular rhetorical relationships can be serialized into a sequence of 
textual units. To the extent that writing follows the normal encoding relationships, a text can be 
parsed to recover (partially) the intended rhetorical structure. There is extensive literature on 
RST analysis of texts and a number of software programs enabling parsing of texts to recover a 
rhetorical structure. A major contributor in this area is Daniel Marcu and his colleagues (Marcu 
1996, 1998, 2000; Marcu, Amorrortu, & Romera, 1999); see also Knott and Dale (1992), Moore 
and Pollack (1992), Sidner (1993), Moser and Moore (1996), and Passonneau and Litman 
(1997). 

Since RST is primarily a theory of how rhetorical structural relations are encoded in text, 
it is to some extent a conflation, in that it handles both the rhetorical representation and the 
marking of the relation in the text in parallel. One of the ways in which natural language creates 
difficulties for the writer is that the same discourse relation may be signaled in many different 
ways or even left implicit. Explicit signals of discourse structure (typically in the form of 
discourse markers such as because, however, and while) are typically polysemous and 
polyfunctional, so that the process of deciding what rhetorical relations are signaled at any point 
in a text is not a simple matter. For reviews of the use of discourse markers to signal rhetorical 
relations, see Bateman and Rondhuis (1997), Redeker (1990), Sanders (1997), Spooren (1997), 
Risselada and Spooren (1998), and van der Linden and Martin (1995). 
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Given that the relationship between rhetorical intentions and their signaling in text 
involves a many-to-many mapping, theories have been developed that focus on the cognitive 
process that presumably drives the decisions needed to control the process of expressing 
rhetorical relationships in text form. These theories are typically concerned with relationships 
that go beyond the explicit signaling of discourse relationships using discourse markers and 
focus on a variety of linguistic cues for text organization that are typically grouped together 
under the general heading of coherence. See Knott and Dale (1992) and Sanders (1997), among 
others. The most important approach is centering theory (Grosz, Weinstein, & Joshi, 1995), 
which focuses on the relationship between focus of attention, choice of referring expressions, 
and the perceived coherence within a text unit. In this perspective, the use of discourse markers 
is one part of an entire array of devices that reflect the cognitive state of the speaker (or the 
anticipated cognitive state of the audience) at a fine level of detail. 

ETS has been involved heavily in the use of tools that identify cues to text structure at the 
level of coherence (Burstein & Higgins, 2005; D. Higgins, Burstein, Marcu, & Gentile, 2004) 
and rhetorical structure (Burstein & Marcu, 2003; Burstein & Shermis, 2003; Burstein, Kukich, 
Wolff, Lu, & Chodorow, 2001; Miltsakaki & Kukich, 2000). The focus of this work has been the 
identification of features that can be correlated with human ratings of writing quality; these 
features contribute to a detailed characterization of productive writing skill embodied in ETS’s 
Criterion writing-instruction service (see also Bennett, 2007). Although discourse expectations 
about such signals can be codified so well that such programs can be successful, it is also 
important to recognize that different genres impose different expectations and to be aware that 
the meaning of particular linguistic and grammatical devices will vary across social contexts. 

The Translation Process: Differences Across Types of Writing 

The process of translating from an original plan to the actual linguistic expression covers 
a range from patterns that easily can be represented as consciously learned templates to complex 
patterns that reflect unconscious linguistic expression. These patterns draw upon different 
resources in long-term memory and are likely to be activated and employed differently in 
different types of writing. In the case of argumentation, an issue that needs to be considered very 
closely is the effect of short-term memory limitations and, more generally, the trade-off between 
various goals in the course of translating from plan to text. Coirier, Andriessen, and Chanquoy 
(1999) developed a sophisticated account of how such trade-offs work, beginning with a high- 
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level description of the task requirements for argumentative text and working their way forward 
to some of the trade-offs that have to be made to compromise among the conflicting demands of 
effective argumentative writing. 

Coirier et al. (1999) began by noting that a persuasive text, instantiating what they termed 
an extended argumentative text, is appropriate only when the following eight elements 
characterize the social and discourse situation: 

1. There is a conflict between different views about the same subject. 

2. The topic’s social, ideological, and contextual status make it debatable in the current 
discourse context. 

3. The author has motivation to solve the conflict. 

4. In particular, this conflict is solved by use of language. 

5. The author has a position or claim to make. 

6. The author can support this position with reasons. 

7. The author is able to argue against the opposite position. 

8. The author can do so by providing counterevidence. 

Coirier et al. (1999) argued that the very nature of this structure entails that extended 
argumentative texts have a complex hierarchical structure that cannot easily be flattened or 
linearized. Thus, the basic task of expressing an argument in written form involves a conflict, or 
trade-off, between the linear, thematic structure of the text and the hierarchical structure of the 
argument. This trade-off in turn entails a whole series of decisions about how to deploy linguistic 
resources that will provide cues enabling the reader to recover the intended conceptual structure. 
This process may be difficult for any form of extended discourse but is particularly likely to be 
difficult in argumentative writing, where many of the necessary discourse cues and devices are 
not mastered until relatively late. 

Coirier et al.’s (1999) analysis focused primarily on the issue of linearizing the text, but 
they also noted considerable potential for working-memory overload due to the need to 
simultaneously manipulate the underlying argument, decide on the best order of the component 
elements, and signal the relationships among the parts of the text using appropriate discourse 
cues. This analysis is consistent with the general literature on sources of difficulty in text, but it 
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suggests that an interaction between working-memory constraints and lack of linguistic skill may 
be a limiting factor on the emergence of effective argumentation and other fonns of complex 
writing. Such limitations will come into play at least to the extent that writing requires mastery 
of relatively complex linguistic patterns (sentence patterns involving subordinate clauses, clause 
sequences making sophisticated use of discourse connectives, and the like). Indeed, the kind of 
analysis presented in Coirier et al. (1999) is a model not just for the analysis of persuasive 
writing, but also for how to model the multiple constraints that characterize any particular genre 
or purpose of writing. In each genre, various constraints, such as the need to linearize the text, 
must be traded off against others, such as the need to handle particular social transactions with 
the audience. 

Revision and Reflection: Critical Thinking and Reading in Different Types of Writing 

Thus far we have considered all elements of the writing process except those involved in 
revision and editing: the ability to read one’s own writing and evaluate for narration, exposition, 
and argumentation. Given the communicative considerations that have dominated our discussion 
thus far—for example, that argumentation is strongly social and interpersonal in its orientation, 
involving what is at least implicitly a dialog with one’s intended audience, and that success in 
argumentation requires an ability to anticipate how the audience might react—we hypothesize 
that the ability to write effective arguments depends essentially upon an ability to interpret and 
respond critically to persuasive writing produced by others. Similarly, we have reason to think 
that the ability to produce high-quality expository prose depends strongly upon being able to 
create a mental model of how another reader would react to one’s prose, and thus on a different 
sort of critical reading, in which what matters is a meta-awareness of how another person is 
likely to react to one’s writing. 

The hypothesis that writing is most effective when coupled with reading and that 
reading—especially critical reading—is an important component of effective writing should be 
explored, but it is not particularly well developed in the literature. Tierney, Soter, O’Flahavan, 
and McGinley (1989) presented evidence that combined reading and writing tasks enhance 
critical thinking, but their guidance did not address directly the extent to which training in critical 
reading enables effective revision and editing of one’s own work. The influence of reading skill 
on academic writing is thus a subject that needs further exploration and review. 
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Prospectus 

All of these considerations together suggest that each stage in the writing process is 
sensitive to purpose, genre, and context, and that it is impossible to speak of the writing process 
in generic terms without considering the specific demands imposed by a particular purpose, 
audience, and context. If we use the contrast among persuasive, expository, and narrative modes 
of writing as a way of illustrating such differences in cognitive demand, we may note that 
persuasive writing presents problems that differ in significant ways from the problems presented 
in writing narrative or even expository prose. Expository prose similarly presents challenges not 
present when students are writing narratives. The planning process may require that writers 
deploy skills in informal reasoning that may not yet be developed. Further, the planning process 
almost always requires writers to perform relatively sophisticated assessments of how an 
audience is likely to perceive their arguments or, in the case of expository writing, to recover the 
intended conceptual structure. Moreover, the process of translating plans into text form requires 
mastery of a whole series of skills for organizing and presenting information or arguments and 
clearly signaling them in the text, which once again may not have been mastered. In this context 
it is worth noting that some of the categories identified by Scardamalia and Bereiter (1986) as 
most difficult for student writers (e.g., generating content, creating high-level plans, controlling 
document structure, and revising text) are arguably among the tasks most critical to producing 
quality content. These considerations support a view of competent writing as complex, integrated 
problem solving, which draws upon a rich repertoire of underlying proficiencies. 

WRITING PROFICIENCY, WRITING INSTRUCTION, AND THE 
CHARACTERIZATION OF WRITING SKILL 

Cognitive Models and Pedagogy 

Thus far, the discussion has been for the most part focused upon the skills brought to bear 
by experienced writers and on the cognitive demands imposed when a writer seeks to perform at 
a high level of competency. It has not addressed the problems novice writers face or pinpointed 
how novice writers are to make the transition to full expertise. These are pedagogical issues, and 
although a full review of the pedagogical literature is not within the scope of the current 
document, the powerful influence of certain cognitive models on instructional practices should 
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be acknowledged. Indeed, a primary goal of the CBAL research initiative is to integrate a deeper 
understanding of cognitive processes into instruction as well as into assessment. 

In historical terms, as briefly noted earlier, the most notable example of this phenomenon 
has been the effect of the work of Hayes and Flower (1980) on writing pedagogy. Traditionally, 
writing instruction emphasized aspects of quality in the completed text, often by asking students 
to analyze exemplar essays. In contrast, the process approach aims to capture the temporal 
complexity of writing by emphasizing the recursive nature of problem solving within the 
activities of composing, prewriting, writing, and rewriting. Since its advent 30 years ago, the 
process approach to writing instruction has become a standard approach in language arts classes. 

During the course of this popularization, the process approach grew to encompass a range 
of assumptions about writing. Olson (1999) identified 10 essential characteristics of the process 
approach: 

1. Writing is an activity, an act composed of a variety of activities. 

2. The activities in writing are typically recursive rather than linear. 

3. Writing is, first and foremost, a social activity. 

4. The act of writing can be a means of learning and discovery. 

5. Experienced writers are often aware of audience, purpose, and context. 

6. Experienced writers spend considerable time on invention and revision. 

7. Effective writing instruction allows students to practice these activities. 

8. Such instruction includes ample opportunities for peer review. 

9. Effective instructors grade student work not only on the finished product but 
also on the efforts involved in the writing process. 

10. Successful composition instruction entails finding appropriate occasions to 
intervene in each student’s writing process. (Summarized in Bloom, 2003, 
pp.32-33) 

According to this conception, the process approach helps teach novice writers that 
writing can involve extensive planning and revising. Students leam to identify the problems that 
define each stage, along with strategies for solving them. 
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The process approach to writing instruction is not without critics, as some composition 
theorists recently have proposed that writing instruction has moved beyond process to 
postprocess. According to Kent (2003), postprocess theorists generally hold that writing is 
public, interpretive, and situated. That is, writing occurs as an interchange between language 
users, both writer and audience (i.e., public), who are trying to make sense of language in a 
particular context (interpretive). Thus, the postprocess theorists shift from a view of writing as a 
matter of psychological and cognitive processes, reflected in the recursive cycles of prewriting, 
drafting, and revising, to a view of writing as culturally and linguistically determined. These 
theorists adopt a critical stance “to address a host of issues from diverse social, multicultural, 
ethical, and other perspectives” (L. Z. Bloom, 2003, p. 31). In contrast to process theory, which 
“proposes a common writing process, about which generalizations can be made” (L. Z. Bloom, 
p. 36), postprocess theory suggests that “no codifiable or generalizable writing process exists or 
could exist” (Kent, 2003, p. 1). This theoretical critique, however, has not (so far) produced a 
new pedagogical approach with equal influence across classrooms at all grade levels. 

Apart from debating matters of theory, researchers have addressed the pragmatic question 
of pedagogical effectiveness: Can certain types of instruction be shown to improve writing 
proficiency? In his review of research on writing instruction from 1965-1985, Hillocks (1987) 
concluded, “The most important knowledge is procedural, general procedures of the composing 
process and specific strategies for the production of discourse” (p. 81), though he also 
emphasized the efficacy of teaching inquiry and evaluation strategies. Another review of 
research in writing instruction revealed significant positive effects for teaching writing strategies, 
such as planning and revision (Graham, 2006). A study of results from the 1992 and 1998 NAEP 
writing assessments (National Center for Education Statistics, 2002) revealed a positive (but not 
necessarily causal) relationship between process-related classroom activities and higher writing 
scores, although the National Center for Education Statistics also noted that various mediating 
factors, such as time spent on tasks, may contribute to that relationship. 

Recently, Graham and Perin (2007a, 2007b) conducted a meta-analysis of research on 
writing. They identified a set of recommended approaches for teaching writing to adolescent 
students. Ordered (albeit with caveats) by descending effect size, the 11 instructional methods 
are the following (Graham & Perin, 2007a, pp. 4-5): 
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1. Writing strategies involves teaching students strategies for planning, revising, and 
editing their compositions. 

2. Summarization involves explicitly and systematically teaching students how to 
summarize texts. 

3. Collaborative writing uses instructional arrangements in which adolescents work 
together to plan, draft, revise, and edit their compositions 

4. Specific product goals is a method in which students are assigned specific, 
reachable goals for their writing. 

5. Word processing uses computers and word-processors as instructional supports for 
writing assignments. 

6. Sentence combining involves teaching students to construct more complex, 
sophisticated sentences. 

7. Prewriting engages students in activities designed to help them generate or organize 
ideas for their composition 

8. Inquiry activities engage students in analyzing immediate, concrete data to help 
them develop ideas and content for a particular writing task. 

9. The process writing approach interweaves a number of writing instructional 
activities in a workshop environment that stresses extended writing opportunities, 
writing for authentic audiences, personalized instruction, and cycles of writing 

10. Study of models provides students with opportunities to read, analyze, and emulate 
models of good writing. 

11. Writing for content learning uses writing as a tool for learning content material. 

Graham and Perin (2007a) cautioned that these recommendations do not, as a set, 
constitute a full curriculum; they also noted that many of these approaches are interlinked rather 
than distinct. What is perhaps most striking, though, is the breadth and variety of instructional 
practices supported by observable results. (Note, for example, that the process approach and the 
more traditional study of models appear next to each other, as potentially complementary rather 
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than mutually exclusive.) Such findings are congruent with the recognition that writing 
proficiency involves a complex array of interrelated factors. 6 

The Transition From Novice to Skilled Writer 

The task of the writing teacher is to enable the student to move toward proficiency, and 
thus it is useful to review the cognitive differences that distinguish novice from expert writers. 
Understanding these differences helps identify the types of changes teachers must induce in their 
students to improve their writing skill. 

Much of the literature reviewed previously has instructional implications and can be used 
to characterize the difference between novice and skilled writers. In particular, skilled writers 
spend more time planning and revising their work than novice writers; they focus more of their 
effort and attention on managing the development of content and concern themselves less with 
its formal, surface characteristics; and they employ a variety of self-regulatory strategies 
(Bereiter & Scardamalia, 1987; Galbraith, 1999; Graham, 1990; Graham & Harris, 2005; 
Kellogg, 1988; McCutchen, 2000; McCutchen et al., 1997). Moreover, novice writers benefit 
from instruction on planning and revision strategies and on thinking about topic-relevant content 
(De La Paz, 2005; De La Paz & Graham, 1997a, 1997b, 2002; Graham & Perin, 2007a, 2007b; 
Hillocks, 1987, Kellogg, 1988; Quinlan, 2004). 

As discussed previously, Bereiter and Scardamalia (1987) characterized the difference 
between novice and skilled authors as the difference between a knowledge-telling approach and 
a knowledge-transforming approach to writing. In a knowledge-telling approach, the focus of the 
writer’s effort is on the process of putting words on the page. Whatever ideas the author is able 
to mobilize are assumed to be good enough; writing takes place as a direct translation of those 
ideas into words, and as soon as the words are on the page, the writer is finished. In a 
knowledge-transforming approach, writing is a recursive process of knowledge development and 
knowledge expression. Planning is more than organizing existing ideas; it is an active process of 
questioning, research, and rethinking. When text is produced, it is not viewed as the final product 
but is subjected to systematic evaluation and revision in the light of a critical evaluation both of 
the content being communicated and its effectiveness in advancing the author’s rhetorical goals. 

Knowledge transfonning is by its nature a much more effortful and sophisticated process 
than knowledge telling, and so it is not particularly surprising that novice writers default to a 
knowledge-telling approach. It is, however, useful to consider in greater detail why authors may 
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fail to use a knowledge-transforming approach to writing, as these reasons suggest instructional 
strategies. The literature suggests five categories of explanation: (a) interference effects 
(undeveloped or inefficient literacy skills), (b) lack of strategic writing skills, (c) insufficient 
topic-specific knowledge, (d) weak content reasoning and research skills, and (e) unformed or 
rudimentary rhetorical goals. We shall consider each of these in turn. 

Interference Effects 

Writing processes compete in working memory. The high-level, strategic skills required 
for a knowledge-transforming approach to writing place heavy demands on memory and 
attention. In many novice writers, the absence (or, more likely, inefficiency), of fundamental 
skills such as oral fluency, transcription, and text decoding (reading) makes it impossible to free 
up the working-memory capacity needed for strategic thought (Kellogg, 2001; Olive & Kellogg, 
2002; Piolat et ah, 1996; Torrance & Galbraith, 2005). The ability to produce text fluently and 
easily depends both upon oral fluency (Shanahan, 2006) and upon basic transcription abilities 
(Bourdin & Fayol, 1994, 2000) and thus can become slow and effortful if any of these 
component skills function inefficiently. Similarly, the ability to monitor and reflect upon one’s 
own writing, which is critical to planning and revision, depends in large part upon aspects of 
reading skill, both decoding and higher verbal comprehension; thus, reading difficulties can 
cripple revision and planning (Hayes, 1996, 2004; Kaufer et ah, 1986; McCutchen et al., 1997). 

Lack of Strategic Writing Skills 

Even skilled writers can be limited by working-memory capacity, so that they cannot 
handle all aspects of the writing task simultaneously. A significant element in writing skill is the 
ability to intersperse planning, text production, and evaluation, sometimes switching back and 
forth rapidly among tasks and other times devoting significant blocks of time to a single activity 
(Matsuhashi, 1981; Schilperoord, 2002). Controlling writing processes so that the choice of 
activities is strategically appropriate and maximally efficient is itself a skill, one that takes time 
to acquire and that novice writers typically do not manage well (cf. Coirier et al., 1999, for an 
application of these ideas to persuasive writing). 
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Insufficient Topic-Specific Knowledge 

Knowledge of the topic about which one has to write is a critical detenninant of writing 
success. All writing models presuppose a critical role for long-term memory in which the subject 
matter of writing must be retrieved, either in a top-down fashion (Bereiter & Scardamalia, 1987; 
Hayes & Flower, 1980) or in a more bottom-up manner (Galbraith 1999; Galbraith & Torrance, 
1999). Those who already possess the knowledge needed to write about a subject are at an 
advantage. Moreover, the kinds of critical thinking needed to pursue a knowledge-transforming 
approach to writing arguably require at least basic topic knowledge to support judgments of 
relevance and plausibility and to support reasoning about content. Thus, it is not surprising that 
topic knowledge is a major predictor of writing quality (DeGroff, 1987; Langer, 1985; 
McCutchen, 1986). 


Weak Content-Reasoning and Research Skills 

The essence of the knowledge-transforming approach is that writing is not viewed as a 
mere expressive act but as part and parcel of a habit of critical thinking in which the act of 
writing serves as the occasion for, and the focus of, a complex form of problem solving. While 
many of the problems the writer faces are rhetorical, having to do with audience and purpose, 
these goals typically require the author to develop ideas; to identify infonnation needed (but not 
possessed); and to obtain that information, whether by observation, inference, argument, or 
research. These skills are arguably among the most important skills needed for academic writing 
(cf. Hillocks’s 1987 meta-analysis, which indicated the critical importance of inquiry strategies 
to improve student writing, and his related arguments in Hillocks, 1995). 

Complicating the picture is the fact that the reasoning required for the successful 
completion of a writing task varies with purpose, audience, and genre. In a narrative writing task, 
for instance, reasoning about actions and motives is likely to be relatively important, whereas an 
expository writing task is more likely to place an emphasis on such skills as definition, 
generalization, analogy, and a persuasive argumentation task on evidence and refutation. Thus, a 
wide collection of content-reasoning skills is needed for effective writing, and individuals may 
be strong on some of these skills and weak on others. However, overall, the evidence is that one 
cannot assume that novice writers, or even all adults, have the strong content-reasoning and 
research skills needed to support a knowledge-transforming approach to writing (Felton & Kuhn, 
2001; Kuhn, 1991; Kuhn et ah, 2004; Means & Voss, 1996; Perkins, 1985; Perkins et al., 1983). 


37 



Unformed or Rudimentary Rhetorical Goals 

In the end, all of the issues surveyed thus far depend upon setting appropriate rhetorical 
goals. The key weakness of the knowledge-telling approach to writing is that it effectively 
assumes a single goal for writing: the expression of existing knowledge modified minimally to 
suit the task. A sophisticated writer must be aware that all writing is communication within a 
social context in which the author must take the audience into account, collaborate with others, 
and more generally act within one or more communities of practice with well-defined 
expectations about the role writing fills within each community’s sphere of action. 

Students evidently benefit from instructional activity that clarifies the intended audience 
and makes the writer’s obligations to that audience clearer (Cohen & Riel, 1989; Daiute, 1986; 
Daiute & Dalton, 1993; Yarrow & Topping, 2001). Activities that appear to have a strong 
beneficial impact on student writing (Graham & Perin, 2007a, 2007b) include those that make 
the act of writing more social and interactive, such as peer review. 

Just as critically, each act of writing and each mode and genre of writing operate within a 
system of social norms and expectations. Students clearly benefit when writing instruction is 
structured to enable students to internalize these social norms (Flower, 1989); Kent, 2003; 
Kostouli, 2005). This idea can be extended usefully by viewing the writing classroom as 
functioning best when it is explicitly designed to enculturate students to participate in academic 
and other writing communities of practice (Beaufort, 2000). 

The key point is that skilled writers have the necessary knowledge and community 
connections to set appropriate discourse goals. Such goals will be salient for students only to the 
extent that the community and the audience are made real and present within their writing 
experiences. Further, such goals will be practicable only to the extent that students have acquired 
the cognitive capacities or skills needed to achieve them. 

The considerations adduced thus far imply collectively that the goal of writing instruction 
is to enable novice writers to surmount barriers to adopting a knowledge-transforming approach 
and to provide them with what they need to leam how to set appropriate rhetorical goals, reason 
appropriately about content, and manage their writing activities efficiently, with minimal 
problems due to inefficiencies in underlying component skills. In effect, the purpose of writing 
instruction is to manage the transition from knowledge telling to knowledge transfonning and to 
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do so as part of a process of enculturating students in the practices and expectations of a literate 
community. 


Schemas, Scaffolding, and Instruction 

We have been developing two central themes: the intrinsically dialogic nature of writing 
and the fact that many people, both children and adults, lack the skills—and, by implication, the 
schemas in long-tenn memory—to engage appropriately with the tasks required of literate 
writers in a literate community. In that context, little separation can be made between writing 
skill and the ability to understand and think about content. Whereas someone might be able to 
reason about content without being able to communicate that understanding, the reverse almost 
certainly does not hold: Becoming a skilled writer and acquiring the set of skills required to 
reason about the content about which one is writing are practically inseparable, as shown by the 
salience of content in the pedagogical reviews cited above. In Graham and Perin’s (2007b) meta¬ 
analysis, inquiry strategies, writing for content learning, and prewriting all intrinsically involve 
reasoning about content. In their meta-analysis, the teaching of writing strategies had the largest 
instructional effect, but many of these strategies can be viewed as supports for verbal reasoning 
about the content of the material being addressed by the writer. The instructional literature seems 
to suggest that these kinds of verbal-reasoning skills are best taught as planning strategies. They 
function in effect as supports, or scaffolds, that simplify the process of thinking about content 
during the writing process. In what follows we shall examine some of the relevant literature in 
the realm of persuasive and (to a lesser extent) expository writing. These sections should be read 
not as an exhaustive review of the pedagogical literature on critical thinking and content 
reasoning but as an exploration of ways to scaffold critical-thinking skills in the context of 
writing instruction. 


Toulmin Logic 

Both the dialogic nature of argument and the importance of appropriate conceptual 
schemata are central to one of the most influential approaches to argumentation, Toulmin logic 
(Toulmin, 2003; Toulmin, Rieke, & Janik, 1984). Toulmin’s approach to logic sets aside the 
traditional, formal approaches to logic in favor of an approach in which arguments are viewed as 
situated in a social context. Once arguments are situated in such a context (as they are in any 
real-life situation), the problem for the argumentation theorist is to account for how arguments in 
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very different settings are structured and to provide a framework in which arguments can be 
compared across domains. This is the purpose of the Toulmin framework. Touhnin postulated an 
abstract structure underlying all arguments, containing one or more of the following elements: 
the claim (what the author is trying to prove), evidence (data supporting the claim), warrant (the 
reason why the evidence supports the claim), backing (the basis on which we believe that this 
kind of argument is credible), qualification (limits on how generally a claim can be asserted), and 
rebuttal (considerations limiting the applicability of a claim or even discrediting it). 

Toulmin logic has had significant impact on educational practice (see, for instance, 
Hairston & Keene, 1981) because it provides an approach to reasoning that can be applied 
generally and is free of much of the technical baggage of more fonnal approaches to reasoning. 
Several lines of application can be discerned. 

One approach involves the fairly direct application of Touhnin as a scheme for labeling 
the parts of a preexisting essay (e.g., taking the Touhnin framework and treating it as a set of 
labels for analyzing the parts of an essay). Such an approach has application both as an analytical 
technique (e.g., for analyzing the detailed content and overall quality of argumentation) and as a 
pedagogical method, involving training students to apply those labels to a text and thereby (to the 
extent the method works) serving as a method for training students in argumentation. 

The application of Touhnin as a method for marking the argument structure of text, and 
by extension as a method for assessing argument quality, provides methods not only for mapping 
the actual structure of arguments in detail, but also for specifying rubrics. For instance, Osborne, 
Eduran, Simon, and Monk (2004) used the Touhnin framework and hypothesized the following 
hierarchy of argument quality: 

1. Level 1 arguments consist of a simple claim versus a counterclaim or a claim versus 
claim. 

2. Level 2 arguments consist of claims with data, warrants, or backings but do not 
contain any rebuttals. There are two subdivisions made here: 

• Level 2A arguments have only a single warrant, piece of supporting data, or 
backing. 

• Level 2B arguments have multiple warrants or backing. 
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3. Level 3 arguments consist of a series of claims or counterclaims with data, warrants, 
or backings and with the occasional weak rebuttal. 

4. Level 4 arguments consist of a claim with a clearly identifiable rebuttal. Such an 
argument may have several claims and counterclaims as well, but they are not 
necessary. 

5. Level 5 arguments are extended arguments with more than one rebuttal. 

This particular approach focuses on the sophistication of the argument—as reflected in 
the choice of developing material. In another approach, Cho and Jonassen (2002) presented a 
rubric that focuses instead on each piece of the Toulmin logic structure and imposes a separate 
measure of quality for each, much like the use in Connor (1990) of Toulmin categories to define 
a rubric of argumentation quality. Connor’s approach, which relies upon a global evaluation of 
the extent to which Toulmin categories (claim, data, and warrant) are developed in the text, does 
not require detailed labeling of each part of the sentence. In this manner Connor’s approach is 
unlike more detailed annotation schemes, such as Newman and Marshall’s (1991), that require 
significant modifications to the Toulmin scheme in order to be able to label the argument roles of 
every piece of a text. 

The base Toulmin argument schemas can be used for pedagogical purposes, as evidenced 
in such argument and reasoning textbooks as Toulmin et al. (1984). However, there appear to be 
significant issues with direct applications of the Toulmin schema when they are performed 
without consideration of how argumentation interacts with the cognitive and social demands of 
the writing process (see Fulkerson, 1996). Hegelund and Kock (1999) outlined some of the most 
important such issues. To begin with, there is the fundamental problem that everyday 
reasoning—and thus writing reflecting the norms of everyday reasoning—usually does not 
explore explicitly all aspects of a Toulmin argument structure, particularly warrants and 
rebuttals. Thus, Hegelund and Kock noted, the direct pedagogical application of Toulmin 
analysis to student papers is likely to lead to frustration, since many of the Toulmin categories 
occur only rarely and one must fill in much of the detail about warrants by inference. A second 
major issue is that the Toulmin (2003) model explicitly requires domain-specific warrants. That 
is, the kinds of reasoning appropriate in one domain (say, legal reasoning) may vary considerably 
from those that are appropriate in another (say, theology), depending in large part upon what 
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shared, social understandings exist in each domain about what kinds of inferences are warranted 
under what conditions. Since beginning persuasive writers (by definition) do not know the social 
expectations about reasoning in specific domains, the application of Touhnin structures without 
adequate grounding is likely to create a situation in which students are being asked to infer 
warrants for which they lack well-developed schemas. Lunsford (2002) noted that this kind of 
context dependence implicitly also involves the need to be able to imagine the audience and 
context to which specific arguments are addressed. Because a warrant is a shared basis for 
reasoning, appropriate warrants can be inferred only if one can identify the audience and 
determine what that audience will be willing to take for granted. 

Another way of making the point is that the Touhnin categories are highly abstract and 
are thus likely to be difficult for students to leam and use. Efforts to teach students to think 
critically often focus on recurrent topoi, or argument patterns, that provide useful tools for 
constructing arguments, even if they can be decomposed into more abstract structures in terms of 
Touhnin logic. This is the approach taken in traditional rhetorics and, along slightly different 
lines, in some methods based on informal logic. Technically, Toulmin’s approach is one variant 
of informal logic; for other models, many of which employ far more concrete argument schemas, 
see Hamblin (1970), Walton (1996), and Tindale (1999). A good case can be made that writers 
may benefit more from having concrete argument schemas that provide them reasonably clear 
models for constructing arguments than from having the more abstract kinds of schemas 
provided in the Touhnin categories. 

Given these kinds of issues, the approach favored by educators who make use of Touhnin 
logic focuses on the use of Touhnin categories during the prewriting and revision phases rather 
than its application to a finished essay. Hegelund and Kock (1999) described their macro- 
Toulmin approach as using Toulmin categories to define the standards for academic persuasive 
writing and applying these categories top-down to structure student expectations about how to 
create effective academic papers by requiring the students to try to cover all the elements of the 
Touhnin schema as they draft and revise a paper. A related approach, that of Mitchell and Riddle 
(2000), simplifies the Touhnin structure to the bare bones of claim, data, and warrant, which they 
presented in terms of the triadic formula SINCE/THEN/BECAUSE (since A, then B, because of 
C). They applied this schema to the prewriting process, using it to engage students in actively 
elaborating an argument rather than leaving pieces of it undeveloped or implicit. 
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Toulmin-oriented approaches, insofar as they focus on planning (prewriting and 
invention), naturally cohere with approaches that view argumentation as an essentially dialogic 
process in which writers must leam to anticipate and engage a critical audience’s response. A 
natural extension of Toulmin-oriented views, therefore, is using Toulmin argument schemes to 
provide scaffolding to enhance reasoning during the prewriting phase. This approach in turn 
leads naturally to approaches in which software tools provide graphical support for activities that 
map out the structure of an argument, again using Toulmin categories, also known as computer- 
supported argument visualization. The scaffolding tools generally combine two elements: 
graphical representation of problem structure and a collaborative problem-solving design in 
which the dialogic aspects of argumentation are preserved in the form of interactions among 
collaborators. 


Scaffolding Through Argument Mapping 

Scaffolding provides an important strategy for supporting argument-related problem¬ 
solving. Cho and Jonassen (2002) presented evidence that scaffolding argumentation during 
prewriting has positive effects on both writing and related problem-solving tasks. Students were 
assigned to collaborating groups who had to solve either well-structured or ill-structured 
problems, and they were provided with tools to support (and record) their deliberations: a 
bulletin-board system and Belvedere, argument-mapping software that enabled collaborators to 
record their arguments using a Touhnin-like notation. (See Toth, Suthers, & Lesgold, 2002, for 
more detail on the Belvedere system.) After training with the software and extensive experience 
using it in collaboration with others, individual students were given an ill-structured problem to 
solve and then were required to write an essay explaining how they went about solving the 
problem and outlining their solution. The resulting essays were scored holistically for the quality 
of argumentation with respect to all five Toulmin categories: (a) claims, (b) grounds, (c) 
warrants, (d) backings, and (e) rebuttals. Statistical analyses (MANOVA and ANOVA) indicated 
that the scaffolding affected all parts of the problem-solving process and caused student essays to 
significantly elaborate claims and grounds, though not warrants or rebuttals. 

Various other argument-mapping systems have been developed, including Mildred, a tool 
used to support reasoning about science, incorporating a Touhnin-based reasoning scaffold (Bell 
& Davis, 2000); ReasonlAble, initially designed as a critical thinking tool, but for which 
pedagogic gains are reported at a variety of levels (cf. van Gelder, 2003), and Araucaria (Reed & 
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Rowe, 2004). These tools share a general family resemblance, in that they use Touhnin 
representations or something similar to structure a visual representation of an argument, typically 
in the context of a dialogic, collaborative interaction. As such, these tools are generally 
compatible with a process-based approach to writing in which invention and revision are central. 

Scaffolding Expository Writing: Rhetorical Topoi, Concept Mapping, and Related Concepts 

The scaffolding provided by argument-mapping systems provides a form of cognitive- 
strategy instruction. There is considerable evidence across many domains, including writing, that 
explicit teaching of strategies, and of the schemas that go with them, is critical to developing 
competence (Graham, 2006; Pressley & Harris, 2006). In particular, there is evidence that 
instruction in appropriate strategies can improve persuasive writing (Wong, Butler, Ficzere, & 
Kuperis, 1997) and other fonns of writing, including expository writing (e.g., Englert, Raphael, 

& Anderson, 1991). Although the relationships employed in Touhnin logic are not applicable to 
expository text, it is not hard to identify conceptual relationships that play a critical role in both 
the organization of expository text and the underlying thought processes. 

In traditional methods of teaching expository text, the schemas associated with expository 
writing are a subset of Aristotelian topoi, including classification, comparison, definition, 
illustration, and cause-and-effect (see Corbett & Connors, 1999). These topoi are usually taught 
as strictly organizational elements, at the paragraph level, as schemata for organizing an entire 
essay, or as methods for invention (e.g., discovery of relevant content). However, the expository 
topoi in fact correspond to fundamental cognitive operations that represent the kind of thinking 
needed to organize knowledge, although the thinking skills involved can be supplemented with 
(perhaps most important) the complex of skills needed to create a good summary, which involve 
judgments of relative importance among a set of interconnected ideas. Researchers appear to 
have paid scant attention to the connection between Aristotelian topoi and thinking skills 
prerequisite to writing. The connection is at least touched on in some early work within cognitive 
approaches to writing (Applebee, 1984; Flower, 1979; Hillocks, 1987), but relatively few authors 
have explored the issues thereby raised. The basic idea is that certain intellectual operations are 
part of learning and organizing knowledge and upon which writing depends, and that writing 
instruction should stimulate such reasoning and inquiry skills. In this area cognitive research 
could make a significant contribution by helping to establish exactly how and when writing 
mobilizes or presupposes specific cognitive capacities for knowledge organization. Lack of such 
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knowledge may account for the uneven benefits observed in the use of writing as a method to 
reinforce learning in so-called writing-to-learn approaches (cfi, Newell, MacArthur, Graham, & 
Fitzgerald, 2006, for discussion and review). 

Expository writing is essentially a method for presenting infonnation textually and as 
such combines all the difficulties of creating a mental model of the infonnation with the 
(sometimes quite high) barrier that everything needs to be put into a single, linear sequence of 
grammatically correct clauses. Many writing teachers recommend that students create a store of 
knowledge to be used in an essay in some nonlinear fonn, as part of the invention or prewriting 
process, whether in the fonn of notes, diagrams, partial outlines, or one of a variety of other 
specific techniques. Hillocks (1987) considered several approaches to writing instruction, 
including inquiry learning, in which prewriting is an active process of developing knowledge 
rather than merely an attempt to generate ideas to put into a text. In Hillocks’s (1987) meta¬ 
analysis, inquiry learning was more than twice as effective as traditional approaches. Since the 
knowledge that is presented in expository text is generally not linear, but forms some kind of 
network of interconnected ideas, a related proposal is that ideas for writing should be developed 
by creating concept maps. There is some evidence that this concept-mapping technique produces 
superior results (Sturm & Rankin-Erickson, 2002), although other studies have indicated 
significant psychometric issues to be resolved (Ruiz-Primo, 2000). 

The key point is the complex relationship between the ability to perform certain kinds of 
reasoning tasks and the ability to create expository (and, of course, persuasive) texts. To the 
extent that writing reflects thinking, students will have engaged in a variety of thought processes 
before they ever begin to write, such as generating hypotheses, creating generalizations, 
marshalling instances, comparing and contrasting ideas, clarifying and defining their terms, 
analyzing structure and causal relationships, and so forth. To the extent that writing is structured 
to require thinking, students’ success at the writing task will reflect the ability to perfonn these 
cognitive tasks easily and efficiently. This presents important challenges when testing writing in 
a context that preserves the essential purposes of writing, which almost always involve 
engagement with and, ideally, critical thinking about the content the writing addresses. 

Towards an Inventory of Skills Differentially Involved in Different Types of Writing 

Given the review of the literature thus far, several key ideas have emerged. In general, we 
have noted the importance not only of linguistic and verbal skills, but also of critical reasoning 
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skills (to deal with content) and of various types of social reasoning skills (since writing is a 
socially situated activity). We have noted that the skills needed for writing success are likely to 
vary by task, and that many of the skills do not develop quickly in novice writers. In particular, 
with respect to persuasive writing, we have noted the following: 

• We have clear indications that skill in argumentation critically depends upon 
developing a level of metaknowledge, of being able to think about arguments rather 
than just argue. 

• We have clear results that creating a dialogic environment, such as between pairs of 
students, increases the quality of an argument (cf., Kuhn et ah, 1997). 

• We have clear evidence that the relevant skills do not develop quickly. Whether we 
are speaking about infonnal reasoning abilities or about the linguistic expertise 
needed to clearly indicate argument structure in a written text, the literature indicates 
that full mastery takes time and is not necessarily complete, even when students enter 
college. 

These considerations strongly support the appropriateness of a scaffolding approach to writing 
instruction in which argument is embedded in a more socially natural, dialogic setting. 

Somewhat different conclusions appear to be supported with respect to expository 
writing: 

• There is clear evidence that high-quality expository writing critically depends upon 
having high levels of prior knowledge about the topic. High levels of topical 
knowledge lead to higher levels of reading comprehension, increasing topical 
knowledge further. 

• It is not so clear that a dialogic environment is crucial, but there is strong evidence 
that the quality of writing is strongly affected by the extent to which the author has 
been engaged with the subject through activities that stimulate thought and inquiry 
about the topic. 

• Many of the skills needed for effective exposition are closely related to those needed 
to take a knowledge-transforming approach to writing; effective exposition depends 
upon actively building (and then communicating) one’s own mental model of the 
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topic, rather than passively accepting whatever organization happens to be present in 
one’s sources. 

• The relevant skills are acquired earlier than are those for persuasive writing, but few 
students (even college students) can be said to have fully mastered them. 

The literature review thus suggests that different combinations of problem-solving skills 
are involved in different forms argumentative and expository writing proficiency, which 
therefore cannot be treated as unitary phenomena. Similar conclusions can be drawn even about 
narrative. Narrative critically involves the ability to model motivations and interactions among 
characters and to infer the causal relations among events. What we end up with, therefore, is a 
picture in which the differences among the traditionally defined modes of writing—and by 
extension, differences across the entire range of writing tasks—depend primarily upon the 
mixture of specific skills and strategies required to achieve the writer’s rhetorical goals. 

The resulting picture is complex enough that it is worthwhile to explore in greater depth 
and to attempt an initial mapping of the types of activities and skills that seem to be involved. 
These are not, by themselves, constitutive of writing skill, and in some cases (such as the 
presence of background knowledge) we may wish not to think of them as writing skills at all, but 
they clearly affect the quality of writing and as such should be examined in detail. 

The following discussion presents an initial outline of abilities that may play a role in a 
proficiency model of writing, even if they are not intrinsically writing skills. By its nature, such 
an outline is incomplete and may present interdependent and interacting elements, but it is a 
critical first step to detennining where research needs to be done to establish an effective 
approach to cognitively based assessment for learning. We shall consider how problem solving 
draws upon four major underlying capacities: (a) background knowledge, (b) verbal-reasoning 
skills, (c) social and evaluative skills, and (d) linguistic and rhetorical skills. After we have 
discussed these issues, we shall consider issues of task and evidence, with a focus on defining 
how to develop an approach to writing assessment that respects the fundamentally integrated 
nature of writing as a skill. 


Background Knowledge 

Prior knowledge is critical to almost any writing task and plays a direct role in a number 
of basic text phenomena, including—most importantly—organization and coherence. A text may 
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follow an organizational plan, yet that organization may be difficult to recognize unless the 
writer has the necessary background knowledge to recognize relationships implied by the text. A 
text may be coherent, but only if the writer has the background knowledge necessary to see how 
one sentence connects to the next. Background knowledge also furnishes critical guidance in the 
prewriting and editing processes, since it provides writers with specific points to develop and 
questions to ask that might not have occurred to them if they knew less about the subject. What a 
writer can do depends in large part upon the richness of that individual’s background knowledge, 
and so our focus will be on proficiencies whose performance depends upon having less or more 
richly elaborated prior knowledge of a topic. 

Background-Knowledge Skills Related to Exposition and Narrative 

We may note at least the following ways in which background knowledge can facilitate 
comprehension by promoting a richer inference-making process, particularly for expository and 
narrative texts. These effects are well established in the general psychological literature, under 
the heading of long-term memory (cf., Ericson & Kintsch, 1994): (a) recognition of relevance, 

(b) recognition of relative importance, (c) retrieval of relevant details, (d) connecting details into 
manageable chunks, and (e) enabling deeper inferential processing. 

Recognition of relevance. One of the first skills enabled by prior knowledge of a domain 
is a simple sense of relevancy. Given a word like anthracite, people with relevant background 
knowledge understand that anthracite is a kind of coal, that coal is dug up from mines, and that 
mines are worked by miners who may belong to unions. Thus people with such knowledge will 
be able to judge not only that anthracite has a strong relationship to coal and mine, but also that a 
topic such as “unions going on strike” has a certain potential relationship to anthracite (if the 
striking members of the union are coal miners and the miners involved dig anthracite). This 
ability to recognize relevance plays a role in reading (where it affects perceptions of textual 
cohesion) and in writing (where it affects what transitions between topics). Note that the ability 
to recognize relevance, by itself, will not guarantee that writers will write coherent prose. They 
may recognize a relation but fail to provide cues in the text that will allow readers to recover that 
relation. Clearly, however, if people lack enough background knowledge to make simple 
judgments of relevance on some topic, they are unlikely to be able to write well about it, if 
indeed at all. 
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Recognition of relative importance. Another key skill enabled by prior knowledge of a 
domain is the ability to recognize which aspects of a topic are important and which are 
secondary. Given the subject of coal in general, and anthracite in particular, someone with 
suitable background knowledge would know that the relationship between anthracite and 
mining is much more direct and important than the relationship between anthracite and labor 
unions. This sense of what is (relatively) important clearly plays a role in reading (where it 
determines what patterns of organization will make sense to a reader) and in writing (where it 
can guide the author in choosing which concepts to treat as central and topical and which to 
treat as subsidiary details). Note that the ability to make this judgment does not guarantee that a 
writer will produce organized prose, but without a clear sense of relative importance derived 
from background knowledge, it is unlikely than an author will be able to make the judgments 
needed to decide on a good organizing plan. This skill is a prerequisite to being able to write 
summaries but is not in itself equivalent to summarization as a writing skill, which involves 
both a judgment about what is important and a whole series of decisions about how to express 
the important parts in a short text. 

Retrieval of relevant details. Background knowledge means that information is available 
in memory to be retrieved and that the necessary connections have been made to support 
retrieval of relevant information. This is, of course, the fundamental role of long-term memory: 
providing a vast store of information to be retrieved and activated when needed. However, the 
implications for writing are significant, because so much of the variance in human assessments 
of writing quality correlates directly with document length (Page, 1966). The length of 
documents, especially when written under a time constraint, gives fluency a disproportionate 
impact on perception of writing quality. Yet, much of this variation in fluency has to do with the 
elaboration of details, which may derive, in turn, from individual differences in richness of 
background knowledge. 

Connecting details into manageable chunks. One of the other well-known functions of 
long-term memory is to organize information into manageable packets that can be retrieved 
together and treated as units rather than being maintained as separate items in short-term 
memory. One implication is that people with deep prior knowledge of a subject store the 
information in preorganized packets. Thus, when they retrieve knowledge for the purpose of 
writing, they do not have to create the organizational schemes for an expository text, because the 
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organization comes with the content (for them). Knowing how the information is organized is no 
guarantee, of course, that they will successfully indicate that structure to a reader. Yet, the 
implication is that higher levels of prior knowledge in general imply more organized thoughts 
about the topic and hence better organization of the ultimate written product. 

Enabling deeper inferential processing. Finally, storage of information in long-term 
memory means that it is accessible to support inference when that same information is retrieved 
and processed. The implication is that, all else being equal, individuals with high prior 
knowledge of a topic are likely to make more, and deeper, inferences about the content than 
individuals with little prior knowledge. In reading, this thesis is consistent with the findings of 
McNamara et al. (1996), who showed that high-knowledge individuals can understand 
incoherent text better and that the effort of making the necessary inferences actually improved 
their memory for and understanding of the material. Presumably, similar benefits accrue to 
writers, since the capacity to make more and deeper inferences should increase the amount of 
information available to the writer and suggest novel ideas and interpretations not available to 
those who process the information content of the text more shallowly. 

The joint implication of all these considerations is that an assessment of an author’s 
background knowledge is likely to be strongly linked with that author’s performance on a 
topically related, expository-writing task. This is an issue likely to be of considerable importance 
in a cognitively based writing assessment, since assessing writing skill in the abstract requires 
finding ways to control for levels of topical knowledge. This is a difficult task, since expository 
writing skill is precisely the ability to communicate prior knowledge to an audience. 

Background-Knowledge Skills Related to Argumentation 

Prior knowledge is of course relevant to argumentation as well as to exposition. We may 
note at least the following ways in which prior knowledge can support effective persuasive 
writing: (a) recognition of plausibility, (b) retrieval of evidence that directly supports or 
contradicts a claim, (c) retrieval of common-sense theories relevant to a claim, and (d) enabling 
deeper inferential processes necessary for constructing more sophisticated forms of 
argumentation. 

Let us consider each point in turn. 

Recognition of plausibility. Background knowledge allows one to make immediate 
judgments of plausibility. If an argument presupposes that a certain fact is true, and that fact is 
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directly stored in long-tenn memory, it is immediately plausible. This is also the case with 
claims that easily can be deduced from what is known. Statements consistent with known fact 
have a certain plausibility that statements that contradict prior knowledge do not have. As 
knowledge of a topic expands in scope and richness, judgments of plausibility are likely to 
increase in subjective conviction and objective probability. This kind of awareness is not in itself 
a writing skill, but it is strictly a prerequisite to any attempt to formulate a plausible argument 
and thus to write a persuasive essay. 

Retrieval of evidence. A second role played by prior knowledge in persuasive writing is 
precisely its capacity to enable readers to retrieve facts known to support or counter a claim. 

(This is opposed to the ability to infer that a fact supports or counters a claim, which requires 
more by way of general reasoning skills.) 

Access to common-sense theories. Similarly, prior knowledge makes available common- 
sense explanations of fact that count as things that people already know (or think that they 
know). Again, the role that prior knowledge plays here is simply providing access to an available 
explanation and does not in itself provide the ability to assess the argument or determine its 
persuasive force. 

Enabling argumentative inference. All of the processes of inference necessary to make an 
effective argument depend upon having access to relevant facts and interpretive schemes. While 
having such facts and relationships at one’s fingertips may not guarantee good writing, it is 
arguably a prerequisite without which persuasive writing is likely to be shallow and superficial. 

The consequence of all these considerations taken together is that it is no easy matter (at 
least, on a cognitive basis) to separate writing skill from prior knowledge (and interest). Our 
argument is essentially that writing presupposes verbal comprehension and verbal reasoning, and 
that they in turn presuppose prior topical knowledge. Thus, to the extent that such knowledge is 
lacking, both reading and writing will suffer. 

Verbal-Reasoning Skills 

The argument that we have presented entails that verbal-reasoning skills must be treated 
as foundational skills for writing, just as effective use of prior knowledge must be treated as 
providing foundational skills for verbal reasoning and verbal comprehension. The nature of the 
reasoning tasks involved in the three genres differs significantly in focus, as discussed below. 
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Reasoning Skills Connected to Narrative 

Narrative discourse by its nature requires all of the elements usually discussed in 
literature classes and analyzed for purely literary purposes, though most uses of narrative outside 
a purely literary setting involve factual narratives, such as newspaper stories and similar 
presentations of everyday events. Each of the elements of a typical literary analysis of narrative, 
such as character, plot, setting, and theme, is a reflection of a human ability to understand social 
scenarios and not only to model the causal event structure of a narrative, but also to relate 
character motivations and perceptions to the events presented. This kind of interpretive reasoning 
involves at the minimum the creation and maintenance of a situation model in which events and 
their interconnections are stored (Zwaan, 2004; Zwaan & Radvansky, 1998) as a kind of episodic 
memory (Baddeley, 2000). Beyond the ability to map the structure of a series of events, an entire 
series of abilities is connected with the ability to create an imagined world and to model 
scenarios of interactions among people within such an imagined world, which is acquired very 
early, typically by 4 years of age (Astington, Britton, & Pellegrini, 1990; Oatley, 1999). Mar 
(2004) reported a wide range of neuropsychological studies showing that closely related brain 
areas are involved in story production and comprehension and that these include areas also 
implicated in episodic memory, script activation, and other mental processes requiring 
comprehension of causal event structures. In addition, a broad range of additional skills is 
associated with literary narrative—metaphor, symbolism, and the like—which go well beyond 
the considerations discussed thus far (see also the discussion of social-evaluative skills in 
narrative below). 

Reasoning Skills Connected to Exposition 

Exposition is a form of informal reasoning where the focus is on presenting information 
rather than overcoming resistance to a contested claim. In particular, the following reasoning 
skills (among others) appear to support expository writing and are presupposed by it: 

• Classification is the ability to detennine what general categories are relevant to a 
specific case and to divide sets of individual entities up into coherent subclasses. 

• Comparison is the ability to determine what features are common and distinctive 
between two individual entities or concepts. 

• Definition is the ability to unpack the meaning of a concept and restate it in text fonn. 
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• Illustration is the ability to identify good examples of a general concept. 

The critical point to note about these and related skills that may be invoked during 
expository writing is that these are thought processes, not organizational patterns. If a student 
mechanically constructs an essay, plugging in an illustration where a high-level outline says one 
belongs, or writes a comparison-contrast essay because that is what the student thinks the teacher 
expects, the student may or may not be demonstrating mastery of the infonnal reasoning skills 
such expository forms are supposed to embody. The important point to note about expository 
writing is that it communicates not lists of facts, but a structured interpretation; successful 
expository writing communicates explanatory reasoning about a topic. There is evidence that 
engaging students’ ability to reason about an expository text improves at least comprehension 
(Pressley, Wood, Woloshyn, & Martin, 1992) and arguably also writing, insofar as it can be 
categorized as writing to learn (Klein, 2000). 

Reasoning Skills Connected to Argumentation 

Argumentation is first and foremost an instance of infonnal reasoning focused on 
establishing contested claims. Thus, the first question we need to ask about students attempting 
to produce persuasive texts is the extent to which they possess the required reasoning skills. 
Reasoning skills of this sort can be viewed essentially as reflective processes in terms of the 
model outlined in Hayes (1996). We can isolate at least the following major components of 
reasoning skill (this list partially follows the categories studied in Kuhn, 1991, though with 
modifications and additions motivated by Toulmin logic): 

• Ability to fonnulate an explanation is measurable in part by the ability to write a 
sentence that accurately presents a thesis or claim. This corresponds to the Toulmin 
category of claim. 

• Ability to elaborate an explanation is measurable in part by the ability to provide text 
that fleshes out the details of the explanation and applies that explanation to specific 
instances or examples consistent with the explanation. In Kuhn’s (1991) terminology, 
this typically involves the presentation of scripts illustrating the particular scenario 
postulated by an explanation. 

• Ability to generate alternative explanations is measurable in part by the ability to 
produce multiple explanations for the same set of facts. 
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• Ability to recognize evidence is to be able to detennine whether a particular fact or 
circumstance supports an explanation or militates against it. 

• Ability to formulate arguments, in Touhnin terms, is to be able to provide data that 
support a claim. Note that formulating an argument in this sense does not entail an 
ability to identify the warrant that justifies the argument, but it goes well beyond 
merely providing illustrations or examples of the thesis. 

• Ability to generate counterarguments involves generating reasons why an argument is 
falsifiable in whole or in part. Generating counterarguments presupposes some ability 
to understand an argument’s warrant without requiring that the reasoner be explicitly 
aware of the warrant. 

• Ability to assess arguments (and counterarguments) entails doing more than coming 
up with arguments for or against. It requires being able to reason about the warrant 
for the argument (even if not explicitly stated) and to be able to identify backing for 
the warrant or identifying ways in which a claim needs to be qualified in order for the 
warrant to make the argument valid. Doing rebuttal of an argument in Kuhn’s (1991) 
sense necessarily involves assessment of arguments, though one equally well could 
assess one’s own arguments. 

This list of proficiencies is designed to capture the results from the literature and also 
from Kuhn’s (1991) studies, indicating that many naive reasoners do little more than generate an 
explanation and flesh it out a bit (cfi, Brem & Rips, 2000). The higher levels of Touhnin’s 
structure (warrant, qualification, backing) are seldom made explicit by reasoners except at the 
very highest levels of proficiency and typically must be characterized as involving what Kuhn et 
al. (2004) termed metacognitive skills. 

The very highest level of skills, being able to assess an argument, requires attention to a 
broad range of typical argument patterns, such as causal reasoning from antecedent to 
consequent, argument by analogy, ad hominem arguments, and so forth. One question that 
appears to be open at this point is whether explicitly teaching schemas for such argument 
patterns is helpful, and, if so, to what extent. 
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Social and Evaluative Skills 


Social and Evaluative Skills Relevant to Narrative 

It may be easy to underestimate the skills required to understand a simple narrative. A 
narrative by its nature is more than a sequence of events. The person listening to a narrative must 
infer the causal structure of the events, and since most narratives involve individual actors 
(people) with goals and motivations, an entire theory of mind (Carruthers & Smith, 1996) and an 
implicit understanding of social interaction are implicitly called into play. There is extensive 
literature on all of these subjects, in literature and elsewhere, and we will not review it 
exhaustively here (but see Abbott, 2002; Adams, 1996; Applebee, 1978; Bal, 2004; Barthes & 
Duisit, 1975/1966; Bonheim, 1982; Booth, 1961; Bortolussi & Dixon, 2003, Brooks, 1984; 
Cortazzi, 1993; Culler, 1975; Emmott, 1997; Fleischmann, 1990; Fowler, 1977; D. Herman, 
2002; L. Herman & Vervaeck, 2005; Hogan, 2003; Holman, 1972; Kearns, 1999; Mandler & 
Johnson, 1977; Margolin, 1989; Mihaelescu & Harmaneh, 1996; Onega & Garcia Landa, 1996; 
Phelan, 1996; Prince, 1973, 1982; Propp, 1968; Rabinowitz, 1987; Ricoeur, 1984; Rimmon- 
Kenan, 1983; Schank, 1995; Souvage, 1965; Stanzel, 1984; Stein, 1982; Sternberg, 1993/1978; 
Todorov, 1981; Toolan, 2001; M. Turner, 1996). 

Let us consider what sort of concepts and tasks are involved, as presented in Figure 2. 
This diagram should not be viewed as anything more than a useful heuristic indicating the kinds 
of concepts and thought processes relevant to narrative reasoning. The key point for our purposes 
is that the reasoning necessary to understand narrative (much less create or talk about narrative) 
is complex, multilayered, and deeply embedded in the mind. 

Social and Evaluative Skills Relevant to Exposition 

The critical social function of exposition is to convey information accurately to another 
person, without the opportunity for questions, clarification, or any of the face-to-face interactions 
that typically help people to gauge whether they have communicated what they intended to 
convey. Thus, the basic social capability that underlies expository writing skill is the ability to 
understand and imagine another person’s information state and to model how one’s own 
communications change that state. There is a well-known set of experiments involving people 
giving directions about how to manipulate an object to third parties whom they cannot see but 
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Figure 2. Diagram of problem-solving tasks for narration. 
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with whom they can converse (H. H. Clark & Krych, 2004). In these experiments, when the 
feedback loop from the audience to the speaker was broken, participants found it difficult to 
provide clear and unambiguous descriptions, and the audience often perfonned a series of actions 
very different than those intended by the speaker. We may postulate a scale of 
decontextualization, marking the extent to which a thinker is able to imagine an interlocutor or 
audience’s knowledge state at increasing degrees of abstraction, along the following lines: 

• The thinker can adjust an internal model of an interlocutor’s knowledge state based 
upon explicit feedback. 

• More abstractly, the thinker can mentally model changes in another person’s 
knowledge states based upon a sequence of information inputs, without feedback. 

• More specifically, the thinker can predict what information another person is likely to 
want next, given a hypothesis about their current knowledge state. 

• Where many different people could be responding to the same information, the 
thinker can imagine more than one interpretation of that information, resulting in 
multiple possible knowledge-states compatible with the same infonnational input. 

The authors of expository texts constantly must perfonn these kinds of mental 
manipulations to adjust what they write to suit an audience. The social element here is 
subordinated to transfer of information but is still very much in play. 

Note the direct connection between the skills needed to understand expository text and 
the skills needed to write it; roughly, the author must be able to take on the mental role of the 
reader and model a reader’s likely understanding of the text—that is, comprehension must be 
wedded to metacognitive modeling. As a result, the literature on comprehension of expository 
text is directly relevant here, though we will not review it in detail (but see Bazerman, 1985; 
Beck, McKeown, Hamilton, & Kucan, 1997; Black, 1985; Chi, De Leeuw, Chiu, & LaVancher, 
1994; Flower, 1987; Graesser & McMahon, 1993; Graesser, Millis, & Zwaan, 1997; King, 1994; 
Kintsch, 1998; Magliano, Trabasso, & Graesser, 1999; Narvaez et ah, 1999; National Reading 
Panel, 2000; Rosenshine, Meister, & Chapman, 1996; Spivey, 1991; Trabasso & Magiano, 1996; 
Zwaan & Radvansky, 1998). 
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Figure 3 is a schema that relates expository writing skills to underlying theory-building 
and theory-communicating skills. A writer’s progression toward more mature thinking and 
problem solving involves both metacognitive control and sensitivity to social context as the 
writer develops more sophisticated strategies for creating, expressing, and revising theories. 
Again, this diagram is intended for purely heuristic purposes to indicate the kinds of concepts 
and tasks intrinsically involved; there are obvious connections to the taxonomy of educational 
goals outlined in B. S. Bloom (1956). 

Social and Evaluative Skills Relevant to Argumentation 

Given the essentially social nature of argument, the ability to model the mental processes 
of other people constitutes another major set of competencies likely to impact persuasive writing. 
Here, the critical competencies are people’s abilities to understand and imagine other people’s 
points of view under varying conditions of distance and abstraction. In particular, the literature 
on children’s acquisition of argument has suggested that the major dimensions of competency 
include, first, basic social competencies, such as the following (cf., Piche & Roen, 1987; Rubin 
& Rafoth, 1986): 

• The person recognizes other points of view when people do not share common 
ground and to identify how their point of view differs. 

• The person understands the pragmatic expectations for argumentation—for example, 
knowing that it is appropriate to advance viewpoints that disagree, that a viewpoint 
must be defended if it is attacked, that premises must be mutually agreed upon, and 
that arguments actually must provide valid evidence for the conclusions being 
advanced. 

• The person determines what information is necessary to enable other people, or an 
audience, to situate an argument in a context that they understand and to negotiate (or 
in more writing-like situations, simulate the negotiation) of a common frame of 
reference. 

• The person can track task relevance of argumentation (i.e., to assess whether it 
satisfies some external goal above and beyond supporting a point of view). 
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Figure 3. Diagram of problem-solving tasks for exposition. 
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• The person can reify from familiar people to an abstract or imagined audience, 
possibly with intervening steps (real and present audience, real but absent audience, 
an audience imagined as if it were a real but absent group of people, or an audience 
imagined as an amalgam of different people with different possible responses). 

• The person can do metacognitive reasoning (e.g., to reason about one’s own 
arguments and discourse). 

More specifically, when an argument is being constructed, necessary skills involve an 
author connecting specifics of argumentation to the audience, including the following: 

• The author anticipates the credibility of a claim or explanation to another person, 
dependent on point of view. 

• The author anticipates the persuasiveness of arguments to another person, dependent 
on point of view. 

• The author anticipates arguments and counterarguments that another person would 
produce in response to one’s own arguments. 

• The author can select and arrange arguments most likely to convince a particular 
audience. This includes both the task of selecting individual arguments and also 
(though this might be treated separately) what Coirier et al. (1999) referred to as the 
process of choosing the best hierarchical arrangement of arguments, a process that is 
intrinsically connected to the anticipated response of the audience. 

There are obvious connections between these skills and general reasoning skills; for 
instance, the ability to generate other people’s arguments and counterarguments obviously 
depends upon the ability to generate arguments for oneself. 

The schema in Figure 4 presents a diagram indicating interrelationships among 
argumentation skills described above. Argumentation is shown as an interactive developmental 
process, a dialectic in which arguments need to be developed, tried out, assessed, and revised in 
light of potential audience response, using socially established schemata. Figure 4, as with 
Figures 2 and 3, is intended entirely as a heuristic that helps to highlight some of the concepts 
and tasks critical to this type of reasoning. We have reviewed much of the literature in this area, 
but for a further overview, see van Eemeren (1996). 
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Figure 4. Diagram of problem-solving tasks for argumentation. 
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Linguistic and Rhetorical Skills 

Even in the simplest cases of verbal argumentation, the ability to generate an argument 
and adapt it to one’s audience depends critically upon linguistic abilities, both interpretive and 
expressive. One must be able to take one’s intended argument and express it to the audience in 
such a way that one’s rhetorical intentions are clear, and one must be able to interpret accurately 
the rhetorical intentions of one’s interlocutors. This ability depends in turn upon mastery of the 
linguistic resources that signal rhetorical intention. Similar observations apply to exposition: An 
expository text also needs clear signals of rhetorical intentions, though the specific devices used 
will vary. We keep the discussion of these devices together, as it is not at all clear that the 
devices used to signal rhetorical structure are particularly different in their requirements in 
argument versus in exposition. We hypothesize the following components of linguistic ability 
that are specifically relevant to academic writing, being careful to distinguish between parallel 
interpretive and expressive variants: 

• The author can perform rhetorical moves, such as concession, justification, and 
elaboration, in a conversational context, involves using appropriate discourse markers 
and other linguistic signals to indicate the rhetorical intent. 

• The author can interpret rhetorical intent for short stretches of text, prototypically the 
text produced in a conversational turn, involves making full use of surface cues to 
rhetorical intent, such as sentence structure and discourse markers. 

• The author can produce rhetorically marked sentence sequences involves producing 
sentence sequences, of about paragraph length, with the order of the sentences and 
linguistic cues, such as discourse markers and coherence relationships, clearly 
marking the intended rhetorical relationships among the sentences. 

• The author can interpret rhetorically marked sentence sequences, which involves 
reconstructing the intended rhetorical relationships among the sentences in a sequence 
from its linguistic treatment. 

• The author can organize document structure to communicate an argument or 
expository pattern clearly includes mastering specific templates such as the five- 
paragraph essay, learning organizational patterns for presenting multiple arguments, 
and deploying other schemes for organizing a text to present an argument. 
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• The author can infer rhetorical structure from document structure involves being able 
to reconstruct the intended argument or the intended expository relationship, making 
full use of the overall structure of a text. One consequence of this ability would be the 
capacity to identify thesis statements and important claims and subclaims in a text. 

• The author can use linguistic markers of textual coherence to construct an appropriate 
interpretation goes beyond the rhetorical structure marking to include all kinds of 
cues that require inferential processes, such as interpreting pronouns and other 
referential elements, inferring connections among clauses based on content and prior 
knowledge, and so on. 

• The author can structure text sequences to make them coherent given appropriate 
prior knowledge is a component of linguistic ability. 

• The author can control stylistic variables to produce texts appropriate to genre and 
audience in ways that make an argument more effective is a component of linguistic 
ability. Of course, such control presupposes familiarity with a variety of genres and 
hence entails development of templates reflecting different text types. 

• The author can control sentence structure appropriately and to produce a variety of 
sentence types from low to high levels of complexity, avoiding grammatical errors is 
a component of linguistic ability. 

• The author can control vocabulary and use it appropriately is a component of 
linguistic ability. 

• The author can control orthographic and linguistic features (so-called mechanics 
features) keeps texts clear and correct. 

As described above, the rhetorical skills are fairly generic, and their connection to 
specific linguistic structures is left vague. One of the issues, with respect to providing a 
cognitively grounded theory of writing proficiency, is to pin down the relative contribution 
various linguistic elements make at differing levels of proficiency, especially in children, where 
the process of acquiring control of the necessary linguistic resources still may be maturing. 

The skills highlighted above are unlikely to be the only ones needed in a full proficiency 
model for writing, but they are arguably among the main abilities likely to differentiate students 
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capable of producing effective academic writing from those who are not. Almost any writing 
task will involve a mixture of these competencies in combination with the kinds of skills usually 
associated with writing as a component of the language arts. Invention—the process of creating 
an argument—may involve more reflective processes of the sort here labeled general reasoning 
skills and relatively little use of linguistic and rhetorical skills, as those are primarily production 
or interpretation skills. However, the moment the author attempts to make any kind of record of 
his or her process of reflection, linguistic and rhetorical skills come into play. Conversely, the act 
of reading a piece of writing critically may seem primarily to involve interpretive processes, but 
if that is all that happens when a piece is read, it is by definition not critical reading and probably 
will not support revision of a piece of argumentation very well. The integration of these abilities 
into a single, finely tuned perfonnance constitutes expert proficiency in the more demanding 
kinds of writing. 

BUILDING A NEW KIND OF WRITING ASSESSMENT: DESIGN CONSIDERATION 

Breadth of Construct, Narrowness of Measurement 

At this point we have completed a detailed survey of the skills needed to perform well on 
a broad range of writing tasks. The dominant impression we have received is complexity. If we 
view writing skill broadly as the full range of skills needed to write well in a variety of tasks and 
situations, then the only conclusion we can reach is that writing requires an extraordinary range 
of skills and above all depends on the successful integration and coordination of those skills. On 
the other hand, if we consider how writing is usually assessed, where it is assessed directly, the 
picture can be much more limited. At least in the K-12, summative-assessment context, a typical 
direct-writing assessment may involve one or at most a few essay-length responses, scored 
holistically on a 4-, 5-, or 6-point scale, in response to very simple, generic questions. 

The relatively narrow focus of many state-level, direct-writing assessments has been 
criticized extensively by Hillocks (2002), who argued that typical state-level, essay-writing tests 
use very general questions and prompts, require little factual knowledge, operate under a strict 
time limit, are scored primarily for adherence to discourse fonn and rules of correctness, and 
tend to deemphasize content. The problem with this, Hillocks (2002) argued, is that students are 
primarily rewarded for fluently producing grammatical text that adheres to a standard template 
and are not rewarded for engaging in those aspects of writing known to correlate strongly with 
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skillful writing. Moreover, when teachers attempt to prepare students to take these kinds of 
writing tests, Hillocks (2002) argued further, they are likely to mirror the worst features of the 
assessment, focusing on form, rewarding students for surface features (such as the five-paragraph 
essay form) and grammatical correctness and paying little attention to content issues, even 
though the instructional literature has indicated that students need strategies for thinking about 
content far more than they need instruction in formal features of writing. 

Yet, the features of essay-writing tests that Hillocks (2002) criticized are driven by 
important test-design considerations. Very general prompts are used in order not to penalize or 
reward students who happen to have rich content knowledge about the subject addressed by the 
essay prompt and thus may have an unfair advantage, whereas strict time limits are driven by the 
exigencies of the testing situation. Thus, there are strong practical and social reasons why writing 
tests have taken the current form. It is important to note, also, that many state assessments 
provide a writing situation, present a specific purpose for writing, and identify an audience for 
students to address, as in this persuasive writing task from the Florida 10 th -Grade Writing 
Assessment Program (Florida Department of Education, n.d.). 

Writing Situation: 

The principal of your school has been asked to discuss with a parent group the effect 

watching TV has on students' grades. 

Directions for Writing: 

Think about the effect watching TV has on your grades and your friends' grades 

Now write to convince your principal to accept your point of view on the effect watching 

TV has on grades. 

Nonetheless, most state writing assessments are constrained in what they are able to 
assess. Considering the nature of these constraints, many derive precisely from the dominant 
social matrix that governs test-taking situations in American society. The problem—to put it as 
succinctly as possible—is a conflict between the social conditions under which skilled writing 
should be learned and under which it is usually exercised and the best social conditions under 
which to administer standardized tests. Effective writing is an intrinsically social act in which the 
writer has goals to achieve and an audience to address. It intrinsically involves a wide range of 
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social and intellectual skills and cannot be reduced to a small set of easily measured behaviors. 
Although one might argue reasonably that skilled writing, like skilled test taking, requires 
skillful handling of an abstract, distance audience (not physically present), there are clear 
limitations to the construct that can be measured under the conditions typical of many 
standardized writing assessments. The College Conference on Composition and Communication 
position statement on writing assessment expresses criticisms of this type and uses them to lay 
forth five guiding principles (as cited in National Council of Teachers of English, 2007): 

1. Writing assessment is useful primarily as a means of improving teaching and 
learning. The primary purpose of any assessment should govern its design, its 
implementation, and the generation and dissemination of results. 

2. Writing is by definition social. Learning to write entails learning to accomplish a 
range of purposes for a range of audiences in a range of settings. 

3. Any individual’s writing ability is a sum of a variety of skills employed in a 
diversity of contexts, and individual ability fluctuates unevenly among these 
varieties. 

4. Perceptions of writing are shaped by the methods and criteria used to assess writing. 

5. Assessment programs should be solidly grounded in the latest research on learning, 
writing, and assessment. 

The critical concern is that writing assessment should assess the full complex of abilities 
characteristic of writers in actual practice and that it should assess writing without causing 
teachers and students to have distorted perceptions of what writing is. 

It is an open question whether better ways to assess writing can be developed. Ideally, a 
writing assessment would provide information about all aspects of writing competency and 
would be structured so that teachers preparing students to take the assessment would set 
appropriate instructional goals. The research reported here is part of a larger effort at ETS to 
create an innovative approach to K-12 assessment. This approach, CBAL, is based upon the 
following three principles: 

1. Assessment design should be driven by what is known from the cognitive and 
instructional literature. 
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2. In particular, we seek to implement the principles of evidence-centered design 
(Mislevy, Steinberg, & Almond, 2003), in which the design of an assessment is 
driven by the ability to construct an evidentiary argument indicating exactly how 
each part of the test provides evidence for an explicit model of student competency. 

3. Critically, we seek to design assessments that have the following critical properties: 

• Tasks primarily should involve constructed-response formats, not multiple- 
choice. 

• Tasks should be valuable learning experiences in their own right, so that teachers 
will find that the best way to “teach to the test” is to teach in line with best 
instructional practices. 

• Assessments should be administered periodically over the school year to allow 
measurement of growth and to allow leeway for useful instructional intervention. 

One of the primary purposes of the review presented thus far is to inform the design 
process for an innovative approach to writing assessment consistent with these design principles, 
since ETS is attempting to design a writing assessment that, under known constraints of time and 
cost, will approximate full-construct representation. This is a long-tenn research project, not 
expected to produce an immediate product but intended to support innovation in both formative 
and summative assessment. One additional constraint, not specified as part of the CBAL research 
initiative but emerging strongly from the literature review, is the importance of seeking to situate 
writing assessment: to build writing assessments so that they reinforce appropriate social norms 
and expectations about writing and communicate clearly to those who take the tests how writing 
fits into the larger academic picture, where acquisition of content reasoning skills is a 
fundamental requirement. 

A Competency Model to Inform Assessment Design 

The CBAL initiative posits that assessment design should be driven by what is known 
from the cognitive and instructional literature. Further, supporters seek to implement the 
principles of evidence-centered design (Mislevy et ah, 2003), in which the design of an 
assessment is driven by the required ability to construct an evidentiary argument indicating 
exactly how each part of the test provides evidence for an explicit model of student competency. 
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As part of the design process entailed by this approach, we developed a competency 
model inspired by the preceding review of research literature and intended to facilitate an 
assessment design that will have direct and positive effect on teaching. According to this model, 
there are three basic strands of writing competence: 7 

1. Strand I is language and literacy skills for writing. Strand I is concerned with being 
able to use Standard English, being able to use basic literacy skills such as reading 
(i.e., decoding and word-recognition skills), and being able to draft and edit text. 

2. Strand II is writing-process management skills. Strand II is concerned with being 
able to manage writing processes strategically to produce as effective a document as 
possible and thus is concerned with the ability to plan and evaluate a document. 
Strand II thus includes the ability to generate content, choose an organizational plan, 
and evaluate where a document fails to meet its rhetorical goals. It also includes the 
ability to use these skills to manage the drafting process to produce a text that is as 
well organized and developed as possible. There is a strong connection between 
Strand II and certain aspects of reading, such as those processes in which rereading 
one’s own text serves as a strategy for managing the executive control of writing. 

3. Strand III is critical thinking for writing. Strand III is concerned with the underlying 
reasoning abilities that enable a text to be substantively as well as formally strong 
and thus enable the writer to solve a wide range of rhetorical goals. Strand III is 
concerned with underlying reasoning abilities and thus touches upon a wide range of 
skills also important in critical reading. As this discussion indicates, we do not 
envisage a strict separation between reading and writing skill at the cognitive level. 
For assessment purposes, writing must be viewed separately, as it involves 
complexities not normally included under the (relatively) receptive tasks involved in 
reading, but there are deep connections with reading that cannot be ignored (for a 
discussion of the issues involved, see Fitzgerald & Shanahan, 2000). 

These strands correspond fairly directly to the major kinds of processes identified in 
Figure 1 earlier in this review. Strand I corresponds to automatic processes in that diagram; 

Strand II, to the strategic processes in that diagram; and Strand III, to the underlying cognitive 
processes. However, the status is different: In Figure 1, we present actual cognitive processes; 
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here, by contrast, we are concerned with defining a construct for the purpose of building tests. 
The chief issue is (a) whether the ability in question is part of the abilities shared by skilled 
writers, and (b) whether clear lines of evidence are available on which to build an assessment of 
individual skill. 

Each strand can be broken down further, though how the strands are subdivided at this 
point should not be equated with a cognitive model. We are concerned, rather, with what we can 
measure, and whether that measurable skill can and should be treated as an aspect of writing 
skill. In line with this thinking, the CBAL competency model for writing subdivides Strand I into 
several component skills, as shown in Figure 5. 



Figure 5. Strand I of a competency model for writing. 
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Figure 5 distinguishes between composing (generating the intended text), transposing 
(generating the orthographic representation of the intended text), and inscribing (actually being 
able to plan and carry out motor activities that put the intended text into a written product). 

These specifically are competencies involved in producing a written text, separate from more 
general language skills such as reading or speaking and understanding spoken English, which 
however must be included as abilities fundamental to (and arguably, prerequisites for) skilled 
writing. More specific competencies are postulated (e.g., vocabulary, sentence control, and 
style) corresponding to aspects of written language production that can be readily measured in 
written texts. 

At this point it is important to clarify several points. Most importantly, Figure 5 (and 
Figures 6 and 7 to follow that present the rest of the competency model) should not be interpreted 
as implying any kind of linear or hierarchical model of the writing process. The actual writing 
activities measured under the competencies in Figure 5 may take place in a highly interleaved, 
interacting fashion, and nothing about the diagram is intended to suggest anything else. Similarly, 
Figure 5 is not intended to provide a box-and-arrow model of cognitive structure, such as that 
provided in Hayes and Flower’s (1980) work. It represents, rather, a method of aggregating 
hypotheses about writing skill under specific headings, where these headings correspond closely to 
certain types of cognitive ability. The competency model only goes to that level of detail at which 
one reasonably may want to report performance. Its purpose, first and foremost, is to support the 
design of an assessment that will take all relevant variables into account. 

For example, Figure 5 very specifically includes several abilities that properly may be viewed as 
inhibitors, as necessary prerequisites to writing, and that one may not wish to treat as component 
parts of writing skill, taken strictly. Foremost among these are the ability to speak and 
understand standard English, the ability to read standard English, and the ability to inscribe texts 
(whether by hand or using a keyboard). In principle, none of these (from an educational point of 
view) would be viewed as part of writing skill as it is usually understood, but each is inextricably 
linked to success in writing. If a person fails to perfonn well on a writing test, the interpretation 
of the score changes fundamentally if it is discovered that the person could not type quickly 
enough to get the answer on the page, or if the person is fundamentally deficient in English 
language or reading skills. One of the issues in test design that the model entails, therefore, is 
that it is important to measure such prerequisite skills, whether directly or indirectly. 
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Figure 6. Strand II of a competency model for writing. 
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Figure 7. Strand III of a competency model for writing. 
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Similarly, we can divide the second strand into strategic skills addressing basic writing 
activities (e.g., planning, drafting, and evaluation of text) with appropriate subdivisions. Note 
that two elements—drafting as well as proofreading and editing—also figure in Strand I, though 
we have chosen to include one of the subskills of drafting (control of text structure) in Strand II 
rather than in Strand I, because of its close connection with text planning and evaluation. These 
are the skills necessary to manage the writing process and by so doing to determine how a 
document should be structured and developed. 

In terms of the concepts covered in the literature review, Strand II addresses all of the 
skills needed to manage the writing process and control the ways in which documents are 
produced, structured, and restructured. Thus, most of the skills covered under strategy instruction 
in the literature review, along with planning and translating skills strongly linked to higher level 
strategic thinking, are grouped together (see Figure 6). 

Here, as before, it is important to view Figure 6 as presenting a specific cut on a 
collection of skills covered in much greater detail in the literature review. Note that Figure 6 
does not include a node for background knowledge, even though the literature review clearly 
established that background knowledge has a strong impact on writing perfonnance. This lack is 
driven by the consideration that content knowledge is not the same thing as writing skill, no 
matter how strongly the two interact. One of the issues, therefore, needing to be addressed in a 
writing-assessment design is how to control for background knowledge, even though it plays no 
direct role in the competency model. Two of the nodes (activate/retrieve information and 
select/organize content) map directly onto activities involving long-term memory and attention, 
both of which interact heavily with background knowledge. 

Note also that Figure 6 does not presuppose that the skills measured are not proceeding in 
tandem with skills from other strands. In fact, given the literature review, it may be better to 
describe Strand II as identifying the types of strategic skills necessary for effective executive 
control of the writing process—but for that very reason, there are intrinsic, built-in connections 
to the other strands. Appropriate strategic control of the writing process must coordinate all of 
the activities involved in drafting and revising texts with the detailed cognitive processing of 
content involved in Strand III and reviewed in great detail above.The critical-thinking skills 
needed for effective writing (Strand III) can be divided into those addressed to social and 
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rhetorical goals and those addressed to communicative content. These skills can be subdivided in 
turn, leading to the submodel shown in Figure 7. 

Once again, it is important to understand what Figure 7 is intended to convey. As part of 
a competency model for assessment design, it identities a set of skills that may need to be 
measured. There is a strong connection with genre—so close, that we almost could make a direct 
identification with traditional modes of writing, identifying explain/hypothesize with expository 
writing, support/refute with persuasive writing, and narrate/describe/predict with narrative 
writing. However, such an identification should be resisted. In terms of the concepts developed 
in the literature review, the nodes correspond to the types of reasoning emphasized in those 
genres. Thus, for instance, the explain/hypothesize node corresponds to the types of reasoning 
described in Figure 3 earlier in this document, the support/refute node corresponds to the types of 
reasoning described in Figure 4 earlier in this document, and the narrate/describe/predict node 
corresponds to the types of reasoning described in Figure 2 earlier in this document. 

Yet, it would be a mistake to assume that the types of reasoning identified in Figure 7 
appear only in one mode of writing or genre. As modes of critical thinking for writing, they can 
be mixed and combined in a variety of ways, depending on the exact requirements of the content 
and the precise rhetorical demands of the writing task. In fact, the model already contains one 
type of thinking not explicitly identified with a genre: the gather/synthesize node, though 
exemplar texts are not too far afield (in research papers and other source-based texts). Other 
types of thinking may need to be represented in the model; for instance, reflective writing 
involves a type of thinking not clearly identifiable with any of the nodes in Figure 7. The first 
critical point about Figure 7 is that it expresses the conclusion from the literature review that a 
variety of critical-thinking skills are inextricably linked with writing skill, must be called upon 
(to varying degrees) when a writer attempts to solve rhetorical problems and generate 
appropriate text content, and must be viewed as part of the construct of writing as it should be 
assessed in school and college. 

There is a second critical point about Figure 7: its inclusion of the social skills, related 
both to role and audience, as part of the writing construct. As the literature review made clear, 
writing must be viewed as a social construct, part of a network of social roles, expectations, 
institutions, and associated activities that (at least in the context toward which writing in school 
is directed) are intimately concerned with knowledge transfonnation, with the representation, 
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transmission, and construction of knowledge and belief. The goal of writing instruction is to 
socialize novice writers into this social context; the goal of writing assessment is to measure 
whether novice writers have been socialized successfully. By implication, the assessment of 
writing should be designed to take the social, purposeful nature of writing directly into account. 

Assessment Principles and Strategies: Outline of a Writing Framework 

The above competency model (as expressed in the three strand diagrams, Figures 5-7) is 
the foundation for design of a CBAL writing assessment currently under development. As noted 
above, other principles are stipulated within the CBAL initiative, which seeks to solve many of 
the problems with standard, end-of-year, K-12 accountability assessments by moving toward 
multiple tests administered over the course of the school year and toward building tests from 
constructed-response questions exemplifying tasks teachers would accept as valuable in their 
own right. Of course, the CBAL assessments also must fulfill the established standards for 
quality and fairness, including psychometric criteria for reliability and validity (as detailed in 
ETS, 2002). 

To meet the above requirements, and to explore innovative ways of promoting as well as 
measuring writing proficiency, the prototype CBAL accountability writing assessments depart 
from many conventional standardized writing assessments in at least three significant ways: (a) 
the design of tasks and test fonns, (b) the sampling methods for gathering information about 
students’ writing proficiency, and (c) the scoring methods. ETS is investigating the proposed 
approaches in collaboration with scholarly experts in writing assessment and with practicing 
educators in order to identify what types of assessment designs, information about students’ 
performance, and ancillary materials are most instructionally useful. 

Task and Test Design 

Standardized assessments of writing proficiency often rely on multiple-choice items 
measuring knowledge of grammar, usage, and mechanics; on discrete essay tasks that present a 
relatively generic type of prompt (intended to minimize the effect of background knowledge on a 
student’s performance); or on a combination of multiple-choice items and discrete essay tasks. 
These approaches can have significant benefits in terms of technical reliability, predictive 
validity, efficiency, cost, and speed of feedback, but they can have drawbacks in tenns of 
construct coverage, face validity, and impact on instruction (however unintended). 
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The prototype CBAL assessments explore the possibilities of alternate approaches that 
may offer some distinct advantages in these last three areas. The most innovative characteristic 
of the CBAL writing-assessment design is the use of scaffolding. Each test includes materials 
and activities that provide initial guidance to students and enable them to display their writing 
skills to best advantage when they are required to produce an extended text. 

• The assessments scaffold planning and drafting by providing reading materials 
(where needed) to guarantee that all students have the information they need to write 
a thoughtful, substantive essay or other piece of extended discourse, rather than 
having to draw exclusively upon their existing background knowledge. 

• Each test presents a multipart project on one topic that provides an overall context 
and purpose for the individual writing tasks. The thematically unified tasks constitute 
a logical progression of different steps leading up to and including a full-length essay 
or similar document. Thus, the assessments scaffold composition by providing 
inquiry and prewriting items designed both to prepare students for the writing task 
and to telegraph what kinds of thought and analysis are necessary parts of the 
extended writing task. At the same time, each task maintains reasonable 
independence from other tasks, so students can succeed on later steps even if they had 
difficulty at the beginning. 

• The extended tasks in particular will provide scaffolding by including planning tools, 
explicit criteria according to which the response will be evaluated, and possibly 
customized prompting for revision. 

Such scaffolding should make the CBAL writing assessments (and preparatory activities) much 
more useful to students and teachers and enable integrating assessment much more effectively 
with curriculum and instruction. 

Essentially, these aspects of the test design work toward two goals: to support instruction 
(because the tasks are more like what teachers need to teach students if they are to become expert 
writers) and to provide more direct evidence about the full range of competencies specified in the 
competency model. Any one of the proposed periodic accountability assessments (PAAs) will 
measure only a portion of the competency model, but allowing multiple assessments over the 
course of the school year allows for more complete measurement. 
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The configuration of the CBAL writing test prototypes particularly facilitates gathering 
evidence for Strands II and III of the competency model. The tasks requiring extended responses 
provide the opportunity for students to display their skills in using strategies to manage the 
writing process (Strand II), as do, to a lesser extent, the tasks requiring responses of a paragraph 
or two in length. While the extended tasks draw upon critical-thinking skills (Strand III) as well, 
the smaller prewriting and inquiry tasks (even, occasionally, postessay tasks) focus primarily on 
Strand III and therefore permit some disentanglement of this strand from the other two strands. 
An advantage of this approach is that the smaller tasks can provide better measurement of the 
critical-thinking skills of lower performing students who may not do as well on the extended 
writing task. 

In general, the format of the tasks does not explicitly focus on obtaining evidence of 
proficiencies in Strand I (reading and writing standard English). As described below, the 
alternate approach envisioned is to gather evidence of these skills by evaluating the text 
produced in response to the writing tasks, rather than by administering multiple-choice items 
indirectly measuring mastery of standard written English. Although the prototype CBAL 
assessment design has many advantages, it also raises some inevitable issues in terms of its 
requirements and psychometric implications: 

• Time: Multiple-choice or essay tests with relatively short time limits can limit how 
well writing proficiency can be assessed. CBAL test designs require more time than a 
single-prompt writing task (current versions are 90 minutes, rather than the 30-45 
minutes possible with a single-prompt test), but they also offer enhanced pedagogical 
value and the possibility of assessing writing in a context that more closely reflects 
the writing skills needed for success in school. 

• Commitment to content: Generic writing prompts suitable for almost any student tend 
to encourage preparation strategies based on memorization of templates and formulas. 
CBAL test designs place a heavy emphasis on critical-thinking for writing, knowing 
how to think about the content, not just knowing how to produce text in general. This 
approach seems more likely to drive instruction in appropriate directions. 

• Connection to reading: Almost any realistic use of writing also involves reading, and 
yet some large-scale assessments avoid use of reading materials as part of a writing 
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test. Skills that at first blush are strategies for reading—such as note-taking, 
highlighting, formulating research questions, and summarization—double as effective 
planning and organizing strategies and facilitate real intellectual engagement with 
content. The CBAL writing tasks are intended to communicate the importance of 
such reading strategies as strategies for writing. This approach seems congruent with 
instruction and educational goals, as writing in a school context is almost always 
engaged with, and directed toward, texts that students read, whether to get 
information, consider multiple perspectives on an issue, or develop deeper 
understandings of subject matter. We expect that some CBAL writing tests will 
require less connection to reading, to support inference about the impact of integrated 
reading tasks on writing, but the overall thrust of the CBAL writing tests will 
encourage an approach to writing with an organic and sustained connection to 
reading. 

• Connection to curricula: Because the CBAL test designs focus more specifically on 
genre-specific competencies and content than is typical for standardized writing 
assessments, they may better reflect actual curricula and learning expectations than do 
highly generic writing assessments. At the same time, this specificity presents 
logistical challenges, given the diversity of standards across states and diversity of 
writing instruction across schools. 

Sampling and Aggregation of Information 
In addition to exploring new types of assessment designs, the CBAL initiative calls for 
expanding the amount and range of evidence provided about students’ writing skills. The 
initiative designers envision going beyond the customary end-of-year assessment and instead 
designing a set of PAAs that give students the opportunity to display their writing ability across 
multiple writing situations, topics, and purposes. The PAAs would be administered on multiple 
occasions across the school year, with interim infonnation made available in a timely way and 
the final overall score based on the series of tests rather than a single end-of-year assessment. 
Moreover, the tests would sample distinct purposes and modes of writing, including persuasive, 
informational or expository, and narrative or literary texts; in addition, the intent is to cover 
different subject areas, such as humanities, social sciences, and science. 
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Like the individual tests, this sampling approach does involve a significant extra 
commitment of time, but such a commitment may be offset by the depth and richness of 
information that can be gained. The sampling approach reflects the CBAL competency model, 
which represents writing proficiency as constellation of skills too complex to be assessed in a 
single test. 


Scoring Methods Based on the CBAL Competency Model 

A further requirement is that the information derived from the tests appropriately reflect 
the CBAL competency model. The proposed strategies include scoring the constructed-response 
tasks analytically, rather than holistically, and presenting score infonnation in accordance with 
the three strands identified in the competency model. 

Many constructed-response writing assessments employ holistic scoring, in which the 
reader or evaluator takes into consideration all elements of a piece of writing and evaluates the 
overall quality of the text to assign a single score. Holistic scoring has many advantages in terms 
of efficiency, potentially high levels of score agreement between readers, and acknowledgement 
that writing skills are highly interconnected. It is not, however, the most useful method for 
providing detailed information or feedback about a piece of writing. More helpful in this regard 
is analytic scoring, in which the reader looks at specified features of the response separately and 
assigns a separate score reflecting the quality of each of these features. An overview of the 
correspondences between strands in the competency model and the categories for analytic 
scoring and for reporting scores is shown in Table 1. 

The proposed strategy for assessing evidence of the three strands in the competency 
model is the following: 

• Strand I. Student-generated text, rather than multiple-choice items, provides a means 
for assessing skills in Strand I (which approximately corresponds to sentence-level 
writing skills). Scoring responses to the constructed-response tasks analytically 
provides information about this layer of the competency model. Basing scores on 
multiple constructed-response tasks within and across PAAs contributes to the 
reliability of the data. As discussed below, automated scoring will make it possible to 
provide fine-grained information about this strand. 
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Table 1 

Scoring Categories for the Summative Cognitively Based Assessments of, for, and as Learning 
Assessment Design 


Strand in competency model 
Language and literacy skills 
(Reading and writing standard 
English) 


Writing strategies 
(Planning, drafting, and 
evaluating one’s own text) 


Critical thinking for writing 
(Reasoning critically about 
content; reasoning about social 
context) 


Category for analytic scoring and for reporting scores 
Sentence-level skills: 

Control of the conventions of written standard English 
(grammar, usage, mechanics) 

Clarity and variety of sentence structures 
Command of vocabulary 

Document-level skills: 

Organization 

Focus 

Development 

Content-related and socially-defined background skills: 

Critical thinking and related genre-specific skills, 
including those pertaining to narrative, exposition, 
persuasion and effective use of research and sources. 
These are to be scored at a level of detail sufficient to 
measure whether students have mastered particular 
abilities, including argumentation, evaluation of sources, 
and many other specific critical-thinking skills 
Mastery of other critical-thinking skills needed to adjust 
rhetorical goals depending on audience and purpose 
Comprehension or mastery of the social expectations and 
norms governing writing, especially writing in an 
academic setting, including an understanding of the 
social roles writers take on (collaboration, review, 
editing, authorship, etc.) 
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• Strand II. Extended responses, and in some cases responses shorter than a full essay, 
provide evidence about students’ writing strategies and document-level skills. The 
scoring rubrics and score reports would reflect this aspect of the competency model, 
which tends to generalize over a wide variety of genres and purposes for writing. 

• Strand III. Emphasizing critical thinking is a key feature not only of CBAL test 
design, but also of scoring. Rubrics and score information focusing on aspects of 
critical thinking, as appropriate by genre, offer opportunities for feedback that are not 
typically provided in writing assessments. Moreover, giving critical thinking equal 
status with the other strands—writing fluency and organizational skills—should have 
a positive effect on instruction. 

Currently, certain aspects of the competency model are difficult to measure directly and 
are not represented in the scoring model outlined above. In Strand I, typing ability, reading 
ability, and English language competency more generally may need to be measured separately as 
precursor skills, or else (possibly) measured using timing and keystroke data from online 
administrations. Similarly, it is very difficult to provide direct evidence whether students have 
appropriate strategies for managing the writing process, and so the scoring model focuses on text 
characteristics strongly connected to successful control of the writing process. Here also, 
behavioral data from online administrations could provide more direct evidence, but a variety of 
scoring issues may preclude using such measures in a summative assessment (though they may 
prove effective measurements for fonnative and diagnostic purposes). Finally, it is relatively 
difficult to measure the social aspects of writing competency. Some assessment (for instance, of 
audience awareness) is planned using focused selected-response and constructed-response tasks, 
but in many cases it is not clear how best to structure the tests to provide direct evidence of the 
social aspects of writing skill. Thus, the test design described above should be viewed as 
experimental, intended to measure as much of the competency model as reliably can be 
measured, while leaving room open for development of new methods and scoring techniques that 
may provide better measurement of those aspects of the competency model that may not yet be 
fully measured. 


81 



Prospects for Automated Scoring 

The availability of automated scoring raises the possibility of scoring writing assessments 
quickly and providing very timely feedback as well as making periodic assessment more feasible 
and affordable. It also has the potential to distort writing instruction in favor of teaching students 
those features of writing most easily scored by machine. However, automated scoring may be 
able to support revision strategies and provide automated scoring of student use of the 
conventions of written standard English. With automated scoring handling this aspect of text 
structure in the background, it will be easier to use human scoring to foreground writing 
processes and critical thinking for writing, skills for which human scoring is most likely to be 
instructionally useful. 

Given ETS’s existing automated essay-scoring technology (Burstein & Shermis, 2003) 
and other existing technologies for automated essay scoring (Shennis & Burstein, 2003), there is 
already a strong foundation for applying automated scoring in the CBAL context. However, a 
sustained effort to adapt these automated-scoring technologies to the CBAL writing competency 
model will be needed. Current automated-scoring technologies primarily address the first strand 
(language and literacy skills), though current technologies are not yet aligned with the CBAL 
competency model and do not cover all aspects of this construct. Preliminary work indicates that 
natural language processing (NLP) features can provide reasonably complete evidence and 
scoring models for this strand of the CBAL writing competency model. The other strands are less 
easily scored using NLP features and will be addressed via human scoring, with automated 
scoring only where feasible. Although it is important to conceptualize automated scoring for 
these skills, there are also greater risks. Strand II and especially Strand III skills involve an 
intrinsic social element, and even if automated scoring can be made to work well, it must 
confront the consequences of essays being written and scored as if they were not intended to 
communicate with a human audience. 

We do not believe that current technology fully supports automated scoring of Strands II 
and III, especially with respect to long essay tasks. This judgment is reflected in the design of 
draft CBAL writing assessments, which include a variety of smaller constructed-response tasks 
designed to provide measurement of specific writing-process management skills (planning and 
evaluation) and related critical-thinking skills. We hope that these smaller tasks can be scored 
partially using automated content scoring. Thus, in the current design, Strand III will be human 
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scored, Strand II will be measured by a combination of human scoring (for longer tasks) and 
human or automated content scoring (for focused constructed-response tasks), and only Strand I 
will be scored entirely by automated means. 

Since the CBAL writing assessment must cover writing across multiple grades— 
ultimately measuring progress through primary and secondary school—it is critically important 
that it be aligned with a developmental model of writing. We intend in the first instance to use 
the developmental dataset from Attali and Powers (2008) to construct a factor analysis based 
upon a broad array of automatically calculated features: in particular, the microfeatures that 
underlie e-rater 8> , combined with a second set of NLP features reported in previous research 
(Deane, Sheehan, Sabatini, & Futagi, 2006; Sheehan, Kostin, & Futagi, 2007; Sheehan, Kostin, 
Futagi, & Sabatini, 2006). Preliminary work indicates that the factors that emerge from such an 
analysis can be aligned naturally with Strand I of the CBAL writing competency model, while 
presenting a reasonable picture of overall developmental trends; thus, a first goal will be to 
elaborate this analysis into a developmental model. We expect to be able to interpret most of the 
factors that emerge from this analysis as providing measurement for specific competency-model 
variables from Strand I. A necessary goal in the second instance will be to validate the resulting 
model. We intend to begin validation concomitantly with analysis of CBAL writing-assessment 
pilot data. 


A Comparison to Other Writing-Assessment Frameworks 

The complex of ideas presented in the immediately preceding sections, while differing in 
emphasis from most existing writing assessments, is not unprecedented. A variety of innovative 
types of writing assessment has been developed in recent years, and some types employ 
approaches relevant to the CBAL writing assessment framework. Without attempting to be 
exhaustive, in this section we briefly review some of these approaches in the light of the 
cognitive literature and in terms of their treatment of the issues outlined in this review. 

General Frameworks 

A number of general frameworks for assessing writing has been developed over the 
years. One of the most popular among teachers is the 6+1 trait model (cfi, Culham, 2003, based 
on the original 6-trait model of Spandel & Stiggins, 1990), which focuses on a relatively small 
set of traits that can be assessed across any type of writing. One of the most influential 
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frameworks has been the 1998 National Assessment of Educational Progress (NAEP) Writing 
Framework (National Assessment Governing Board [NAGB], 1998). Likely to be equally 
influential is the newly released 2011 NAEP Writing Framework (NAGB, 2007). 

The 6+1 trait model focuses on the following seven specific attributes of student writing: 
(a) ideas, (b) organization, (c) voice, (d) word choice, (e) sentence fluency, (f) conventions, and 
(g) presentation. The model is focused on how to assess an actual piece of writing, in other 
words, on identifying what about a final written product makes the reader react to it as quality 
writing, regardless of purpose or genre. As a result, the framework contains relatively little about 
many things covered in this literature review. The detailed breakdown of thinking-for-writing 
skills developed in Strand III of the CBAL competency model is covered (in a very general way) 
by the 6+1 trait model category of ideas. Practically everything else in the model falls under the 
CBAL competency model’s drafting node, and it is almost possible to identify them one to one 
with specific subnodes of that competency. Organization corresponds to focus/control text 
structure, word choice to use written vocabulary, sentence fluency to control sentence structure, 
conventions to the two nodes transpose (having to do with mechanics and spelling) and 
edit/proofread (having to do with grammatical correctness and ability to correct a text to 
conform with conventions), and presentation partially to the inscribe node (though other aspects 
of presentation having to do with social expectations about what published texts should look like 
may not fit under this category.) The voice category corresponds roughly to the use written style 
node, though the term voice emphasizes the ability to control style to produce one’s own unique 
personal style, whereas the CBAL competency model variable is more focused on whether 
students have grasped basic stylistic expectations appropriate to particular genres, occasions, or 
audiences. 

The basic difference between the 6+1 writing framework and the CBAL framework 
presented above is that the CBAL framework is concerned primarily with student 
competencies—with identifying the skills students need to write well—and less directly with 
identifying the traits that characterize good writing directly, which is the focus of the 6+1 trait 
model. As a guide to how to assess specific traits of writing, the 6+1 model has many 
advantages, but it effectively deemphasizes the critical-thinking skills emphasized in the CBAL 
model, subsuming them under a single heading alongside several language-based traits such as 
word choice, sentence fluency, and conventions. The role of writing processes is also somewhat 
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deemphasized, insofar as there is no direct assessment of whether students have the skills needed 
to revise, critique, or otherwise deal with writing as a complex, recursive process. 

The 1998 NAEP Writing Framework (NAGB, 1998) was intended to establish the design 
of the writing assessments for the NAEP, but it reflected a series of specific decisions about what 
should be measured in writing and thus reflected a position on what aspects of writing matter for 
instructional purposes. Major emphases of the 1998 framework included the following (NAGB, 
1998, p. 5): 

• Students should write for a variety of purposes: narrative, informative, and 
persuasive. 

• Students should write on a variety of tasks and for many different audiences. 

• Students should write from a variety of stimulus materials and within various time 
constraints. 

• Students should generate, draft, revise, and edit ideas and forms of expression in their 
writing. 

• Students should display effective choices in the organization of their writing. They 
should include detail to illustrate and elaborate their ideas and use appropriate 
conventions of written English. 

• Students should value writing as a communicative activity. 

These emphases reflect important features of writing competence as reflected in the literature 
review: the need to cover multiple genres (and the types of thinking that go with them), the need 
for writing to be socially situated, and the need for it to reflect a sophisticated grasp of strategies 
to control the writing process. Similarly, the NAEP specifications indicate potential features that 
may be manipulated to yield different writing tasks, and the list provided is very similar to those 
that could be derived from this literature review (NAGB, 1998, p. 11): 

Discourse Aim 

Major aim—narrative, informative, persuasive 

Subgenre—for example, position paper, story, letter 

Topic 

Information source— personal experience, school, new information 
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Familiarity 

Interest 

Cognitive Complexity 

Recall/Summarize 

Analyze 

Infer/Interpret 

Evaluate 

Audience 

Known/Unknown 
Adult/Child 
Novice/Expert 
Friendly/Unfriendly 
Presentation Format 
Written 
Pictorial 

Evaluation Criteria 
Administration Conditions 
Writing Procedures Measured 
Prewriting/Planning Drafting Revising Editing 

The 1998 framework (NAGB, 1998) suggests an (implied) trait model reasonably similar 
to the 6+1 trait model, with an emphasis on organization, development of ideas, mature language 
and style, adherence to Standard English conventions, and so forth. It also, however, suggests 
genre-specific rubrics that focus attention on the special requirements of narrative, expository, 
and persuasive text. In practice, however, the committee-approved NAEP guides are very similar 
across modes, save the persuasive guide, which requires a position and support for that position. 
In general, the resulting framework is compatible with that outlined in this literature review, 
despite significant differences in the role of critical-thinking skills, their importance in the 
overall rubric, and the overall intent. In particular, the CBAL writing-assessment framework 
departs significantly from the 1998 NAEP model in requiring ample supporting materials to 
socially situate tasks and facilitate student engagement, in its use of scaffolded tasks, and in its 
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emphasis on promoting critical-thinking skills and providing assessment tasks that can be used 
effectively also for fonnative purposes in classroom settings. 

The 2011 NAEP framework (NAGB, 2007) reflects very similar concerns to those 
motivating the CBAL writing assessment, but in many of its particular emphases it differs. In 
particular, the NAEP 2011 Writing Framework is intended to foster an assessment that will 
encourage students to write for multiple and clear purposes and for clear audiences, while also 
setting the following goals (NAGB, 2007, p. 10): 

• To encourage student writers to move beyond prescriptive or formulaic approaches in 
their writing 

• To assess grade 8 and 12 students’ writing using word processing software with 
commonly available tools 

• To measure students’ ability to respond to a writing task in an on-demand scenario 

The CBAL writing framework also is focused on moving students beyond prescriptive 
and formulaic approaches to writing, but the latter two goals are specific emphases for the 2011 
NAEP assessment, reflecting the desire to present timed writing tests in computerized form in an 
environment as much like a normal word-processing environment as possible. The CBAL 
writing framework is intended also to be computer delivered but to allow sufficient testing time 
for a less intensely on-demand writing situation than envisaged in the NAEP framework. 

One key difference between the design of the CBAL writing assessments and the NAEP 
framework lies in the role of critical thinking, genre and purpose, audience, and the use of forms 
appropriate to specific genres. In the 2011 NAEP framework (NAGB, 2007), as with the 1998 
NAEP framework (NAGB, 1998), there is an emphasis on narrative or experiential, expository, 
and persuasive writing, viewed in the 2011 framework as three primary purposes for writing. The 
2011 framework specifies that all tasks should have a clearly defined audience and purpose, 
while leaving room for experimentation with prompts that either specify or leave open to the 
writer the choice of specific forms and specific approaches to thinking and writing. In particular, 
the framework requires forms to be specified at Grade 4 and indicates that at pilot for Grades 8 
and 12, various types of task will be considered, some specifying form, some suggesting forms, 
and some not specifying form, with operational choices being made from pilot-study results as to 
which approach is most effective with which prompt type. 
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The role of approaches to thinking and writing particularly bears comment. The 2011 
framework explicitly recognizes the importance of critical thinking in the following language 
(NAGB, 2007): 

When given a purpose and audience for writing, writers must decide how to develop and 
organize their ideas to achieve the demands of the task. Defined by various composition 
theorists as thinking and writing approaches or problem-solving strategies, such 
techniques allow writers to develop responses of depth and substance (Claggett, 2005; 
National Writing Project & Nagin, 2003; Flower, 1993). Some approaches commonly 
used to develop and organize ideas in effective written communication include analyzing, 
describing, evaluating, and narrating. By using these and other approaches to thinking 
and writing, alone and in combination, writers have considerable flexibility for the 
development and organization of a text. 

While writing tasks on the 2011 NAEP Writing assessment will not specify the use of 
particular approaches to thinking and writing, tasks will be designed to encourage 
students to draw upon a wide variety of approaches to support the development and 
organization of ideas. Responses will be evaluated for the effectiveness of writers’ 
development and organization of ideas in relation to purpose and audience, (p. 12) 

There are critical differences between this approach and the approach to be explored by 
the CBAL writing assessment. The 2011 framework (NAGB, 2007) essentially subordinates 
critical thinking to text organization; it places most assessment emphasis on the organization and 
development of ideas in the final text. What the framework terms approaches to thinking and 
writing are subordinated under the general heading of the development of ideas. The CBAL 
writing-assessment framework, by contrast, specifies that a range of critical-thinking-for-writing 
skills should be assessed directly and that writing tasks should be scaffolded in such a way as to 
enable identification of whether students have mastered the critical-thinking skills needed to 
write well. 

Another way to see the similarities (and differences) between the 2011 framework 
(NAGB 2007) and the CBAL writing-assessment framework is to consider what will be scored 
and reported. Like the CBAL writing-assessment framework, the 2011 framework is a three- 
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strand model, though the strands are not quite equivalent. In the 2011 NAEP framework, scoring 
and reporting focus on the following criteria (NAGB, 2007, p. 43): 

• Development of ideas is effective in relation to the writer’s purpose and audience. 

• The depth and complexity of ideas are effective in relation to the writer’s purpose and 
audience. 

• Approaches to thinking and writing (e.g., analyzing, synthesizing) are used 
effectively in relation to the writer’s purpose and audience. 

• The details and examples used to develop ideas are specific and effective in relation 
to the purpose and audience. 

• Organization is logical in relation to the writer’s purpose and audience. 

• Text structure is logical and effective in relation to the writer’s purpose and to the 
approaches to thinking and writing that the writer has used. 

• Coherence is maintained within and between paragraphs. 

• Focus is maintained throughout the response. 

• Language facility and conventions support clarity of expression and the effectiveness 
of the writing in relation to the writer’s purpose and audience. 

• Sentence structure is well controlled and sentence variety is appropriate for the 
writer’s purpose and audience. 

• Precise and appropriate word choice supports clarity of expression and enhances the 
presentation of the writer’s ideas. 

• Voice and tone are effective in relation to the writer’s purpose and audience. 

• Grammar, usage, and mechanics (capitalization, punctuation, and spelling) support 
clarity of expression and enhance the presentation of the writer’s ideas, (p. 43) 

These criteria can be roughly mapped to the CBAL strands, but with critical differences 
as noted below. Strand I corresponds to language facility and conventions almost exactly. The 
rest of the NAEP 2011 criteria (NAGB, 2007) correspond roughly to Strand II in the CBAL 
writing-assessment framework, insofar as Strand II incorporates both organization and 
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development without regard to specific achievements in critical reasoning. The CBAL writing 
assessment differs precisely in targeting approaches to thinking and writing as a strand in its own 
right, with strong emphasis on mastery of the necessary thinking skills needed to approach 
writing from a knowledge-transforming rather than a knowledge-telling point of view. This 
difference, however, is driven in large part by the fact that the NAEP framework is intended to 
be a low-stakes assessment that reports on-demand writing performance of children only by 
subgroup. Any attempt to measure critical-thinking skills requires a rich and complex task 
structure that goes significantly beyond an on-demand paradigm for writing. 

Certain Other Writing Assessments 

The approach to assessment advocated in this review also has partial precedents in some 
of the writing frameworks established for major standardized tests at the college level, including 
preliminary work that led to the new Test of English as a Foreign Language™ (TOEFL® ) 
Internet-based test (iBT) as well as similar work embodied in writing tasks for the Graduate 
Management Admission Test (GMAT), the Graduate Record Examinations® (GRE®), and the 
Law School Admissions Test (LSAT), among others. The GRE, GMAT, and LSAT, for 
example, have task types that require significant engagement with critical thinking, in particular 
argumentation, such as issue and argument prompts that require writers either to develop their 
own argument or to critique someone else’s argument (Breland, Carlton, & Taylor, 1998; 
Rosenfield, Courtney, & Fowles, 2004; Schaefer, Briel, & Fowles, 2001). While these 
approaches to writing focus on relatively self-contained writing prompts, they represent an 
important emphasis on writing tasks where reasoning and critical thinking play a key role. 

The TOEFL Writing Framework (Cumming, Kantor, Powers, Santos, & Taylor, 2000) is 
particularly significant in that it represents an in-depth rethinking of how writing should be 
assessed in an English language learning context, taking the importance of communicative 
competence into account, and employing a research-based model of writing. It incorporates 
several innovative design elements: 

• Multiple writing tasks cover multiple communicative purposes such as summary, 
persuasion, and description, corresponding to tasks students actually might be asked 
to perfonn in an academic setting. 
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• Interdependent writing tasks are included, where writing is integrated with related 
skills such as reading, listening, and speaking, in addition to relatively independent 
writing tasks. 

• Writing tasks are situationally embedded in academic contexts. 

• Task designs are intended to improve washback in English as a second language and 
foreign language instruction programs. 

The model for TOEFL writing evaluation distinguishes two general classes of evaluation 
criteria: those that fall under a text characteristics model, and those that fall under a reader- 
writer model. The text characteristics model includes discourse and ideas (organization) and 
language use (vocabulary, handling of paraphrase, quotation, and related elements, syntactic 
structure, and grammar/usage/mechanics). The reader-writer model focuses on specifying, for 
any given task, the characteristics of an ideal reader and the extent to which that reader might 
care about any given trait in the written product and in making sure that these expectations are 
clearly communicated to the writer and used to score each prompt appropriately. Ultimately, the 
traits evaluated are described in fairly standard terms. The rubrics used to evaluate responses 
with respect to discourse and ideas focus on organization, coherence, progression of ideas, 
development/specificity/quality of information, and accuracy. The language-use rubrics, by 
contrast, focus on vocabulary and idiom use; handling of discourse connections between parts of 
the text; handling of morphology and syntax; and handling of spelling, punctuation, and other 
text conventions. Note that here, as with most of the other major college-level assessments 
reviewed above, writing assessment focuses on organization, development, and language use, 
even when critical thinking for writing is explicitly invoked. 

The TOEFL Writing Framework is a relatively early example of a writing test designed 
to cover key aspects of communicative competence, and like the CBAL writing-assessment 
framework it embodies a strong concern for realistic, situated tasks. In addition, it is possible to 
identify a number of assessments where considerable efforts have been made either to develop 
new assessments or to improve existing assessments by addressing many of the issues with 
which this literature review has been concerned, including in particular an emphasis on the 
writing process and on the integration of critical thinking and writing. 
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One particularly important antecedent to the approach proposed here is portfolio 
assessment (Camp, 1982, 1985; Elbow & Belanoff, 1997; Wolf, 1989; see also the discussion in 
Eliot, 2005, pp. 213-216), an approach that has been carried out on a large scale in some state 
assessments, including Kentucky (Koretz, 1998; Stecher, 1998). Key to portfolio assessment 
approaches is an emphasis on selecting a wide range of student work, drawn from authentic 
writing tasks. However, in practice, portfolio assessment has presented serious issues connected 
with difficulties inherent in using highly varied (and nonstandardized) work from a variety of 
situations outside a controlled assessment context. While the approach proposed for the CBAL 
writing assessment shares key concerns with portfolio approaches—most critically, a desire to 
sample broadly across a wide range of writing types and rhetorical purposes—we hope to create 
assessments with sufficient standardization built into the design to overcome many of the defects 
of portfolio assessment. 

In the late 1980s, ETS developed the Tasks in Critical Thinking to assess inquiry, 
analysis, and communication skills in the context of a broad academic area: the humanities, the 
social sciences, or the natural sciences. Each task presented a problem to be addressed, relevant 
documents and infonnation to consider, and a series of 8-10 short-answer questions to help 
students think through the problem. As a final exercise, they had to write an essay or report 
based on the information they used to answer the questions. The assessment was performance 
based, with no multiple-choice questions. Scores were reported not by task but rather by the 
critical-thinking and writing skills defined in the framework (e.g., gathering information, 
analyzing information, and presenting infonnation), and institutions used the results for 
outcomes assessment. Although the Tasks in Critical Thinking were highly praised for their 
relevance to good instruction, they were eventually discontinued because not enough schools 
purchased the test. Reasons might have included timing (90 minutes per student) and the 
emphasis on group rather than individual scores. Scoring, either by ETS or locally, also could 
have been a factor. 

The English Placement Test (EPT), developed by ETS for the California State University, 
provides another, more tempered but also more enduring example of how critical-thinking skills 
have entered the realm of writing assessment. In 2002, EPT writing prompts changed from very 
brief questions about personal experience to more issue-based arguments that students need to 
discuss and more emphasis on reading of source materials. Similarly, the EPT scoring guide was 
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revised, placing greater emphasis on reasoning skills than on fluency. The test is taken by high 
school seniors admitted to the California State University system, thus sending a message to high 
school students about the kinds of thinking and writing skills that are required for success in 
college. 

Similar advances in integrating critical-thinking and writing skills can be seen at the 
secondary and elementary school levels as well. In 2002, the Lawrenceville School in New 
Jersey collaborated with ETS to develop a measure of students’ ability to analyze arguments and 
to construct arguments as well as to convey meaning clearly (Mary Fowles, personal 
communication, November 2007). Although state assessments rarely place such emphasis on 
critical thinking, many administer prompts that present two sides of an issue and require students 
to take a position on the issue, making a convincing argument to support their position. The 
rubric rewards students for the quality of their ideas and line of reasoning, which in the best 
responses may include (and effectively dismiss) counterarguments. 

Other advances in writing assessment reflect the role process in writing cognition theory. 
Before writing their essays, students consider several questions designed to help them think 
about the topic and presumably plan a better essay. After writing their essays or other type of 
composition, they review a checklist tailored to the writing assignment, as illustrated in this 
excerpt from Michigan’s Grade 4 English Language Arts Assessment released in 2006 
(Michigan Educational Assessment Program, 2006, p. 17): 

• Do I have a clear central idea that connects to the theme? 

• Do I stay focused on the theme? 

• Do I support my central idea with important details/examples? 

• Do I need to take out details/examples that do NOT support my central idea? 

• Do I use a variety of words, phrases, and/or sentences? 

• Have I spelled, punctuated, and capitalized my writing to help readers understand it? 

The writing traits in this checklist correspond to criteria in the scoring rubric, and in 
Michigan as in many other states, teachers are encouraged to use these rubrics with their 
students. These kinds of changes have brought writing assessment closer to instruction and to the 
general understanding of writing as a complex process that can be taught. 
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Although these various antecedent approaches to writing instruction share much with the 
proposed CBAL writing-assessment framework, it is important not to lose sight of the ways in 
which that framework is unique. Whereas any one element from the following list can be found 
in existing tests, there is little precedent for the combination of elements to be explored in the 
CBAL writing assessment, including in particular the following four: 

1. The use of a project organization for the test guarantees that each task is situated 
within a realistic social context and task structure. 

2. The use of supporting materials for students to read is a method of “leveling the 
playing field” with respect to background knowledge and providing enough 
information that students can be given interesting and challenging tasks. 

3. The scaffolding of tasks within the project organization means earlier tasks are 
designed to make it easier for students to perfonn effectively on more complex 
writing tasks. 

4. Assessment of critical thinking skills for writing is part of the writing construct. 

This last point is critical. To the extent that writing assessment is sensitive to, and 
supports the teaching of, critical-thinking skills, it indirectly supports a model of 
writing instruction that makes content central. 

CONCLUSIONS—AND FUTURE DIRECTIONS 

The approach outlined above was presented to a national advisory panel of writing 
experts and teachers in the spring of 2007. Sample tests were developed and reviewed both by 
the advisory panel and by teachers and school administrators in a selected state and district. Thus 
far, the preliminary responses have been positive and enthusiastic. Although the approach has 
been endorsed by the teachers and writing experts who have seen the sample tests, it is of course 
necessary to find out how well the approach can work in terms of (a) being replicable across 
many different test forms, (b) measuring the targeted skills in the competency model, (c) 
reflecting learning standards and actual curricula, and (d) meeting accepted psychometric 
criteria. The only way to answer these questions is to write additional such tests, administer 
them, score them, collect feedback on the tests from teachers and students, and analyze the 
results. The literature review and competency model presented in this review thus represent only 
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the first step in creating writing tests where task format and content support a richer writing 
construct, one more in line with the desires of writing experts and instructors than many current 
assessments. 

One of the most important contributions this approach may make, if successful, concerns 
the pedagogical effect, however inadvertent, of contemporary high-stakes writing tests. Such 
tests often rely on multiple-choice items covering grammar, usage, and mechanics or make use 
of constructed-response topics that—in an attempt to avoid measuring background knowledge— 
may be relatively unsubstantial or content free. These choices are driven by the constraints of a 
high-stakes testing situation, and may be—at least in the current state of the art—a choice 
necessitated by the need to maximize reliability under testing conditions that allow very little 
time to assess writing. Yet, writing assessment—and writing instruction—should allow and 
encourage students to apply writing skills to substantive, meaningful infonnation and ideas. It is 
not clear, even so, how to accomplish these goals while meeting many of the other constraints for 
summative assessment, since existing assessments can offer significant benefits in terms of cost, 
technical reliability, predictive validity, efficiency of administration, and immediacy of feedback. 
Conventional multiple-choice, essay, or combined-format tests thus may address important 
requirements for feasibility and defensibility, but there is room for improvement in the way 
writing assessment impacts writing instruction. It is thus important to explore how to provide an 
approach to writing assessment that can meet goals of validity, reliability, and feasibility while 
supporting a more substantive approach to writing. 

Again, the initial enthusiastic reaction on the review panel suggests that the very different 
approach that the CBAL initiative proposes could have a markedly positive effect on instruction 
and learning. Using a project approach that incorporates resource documents has many benefits: 

It builds in an emphasis on purpose and context, as provided by the overarching structure and 
individual tasks, and promotes deeper engagement in a topic. The approach not only offers a 
means to measure both integrated and discrete skills, but also provides an opportunity to assess 
skills not commonly covered in writing tests, such as gathering and synthesizing infonnation or 
reflecting on ways in which one might present certain content differently depending on audience. 

Indeed, each CBAL writing assessment is designed to permit the student to think and 
learn more about the topic. By doing research and critical thinking within the framework of the 
test, as well as producing one’s own text, the student leams incidentally about content but more 
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fundamentally about the habits of mind that contribute to proficiency in writing. The creation of 
formative tasks that mirror the summative tasks (one of the objectives of the proposed work) will 
help promote this process. That is the hope and expectation behind the 2008 CBAL writing 
initiative. However, these hypotheses require the confirmation, or reformulation, that only can 
result from collecting a larger body of information than is presently available. 
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Notes 


1 Note, incidentally, that much recent research on writing and writing instruction has been 
inspired by the process-writing approach stemming from Hayes & Flower, 1980, which is 
reviewed in detail in another section in this report. The relation between process approaches 
to writing and mode/genre will be discussed then. 

' We should distinguish, in this context, between the relatively unanalyzed act of producing 
everyday narrative, which is deeply embedded in ordinary social interaction, and the far more 
demanding types of writing necessary to produce literary narratives. In linguistic analysis 
those things that seem simplest and most straightforward often reveal the greatest complexity 
when analyzed in depth. The extent to which narrative is cognitively and socially embedded 
into language from a very early stage is a theme that will recur throughout this review, but 
without implying a denial of the complexities involved. 

3 

Whereas these approaches recognize the role of research, and the importance of the thought 
processes involved in formulating questions and obtaining information by reading external 
sources, their emphasis is on the way information is processed and used to set goals, which 
depends upon information already present in long-term memory to guide retrieval and 
strategy formation. 

4 In fact, one of the complications of a monologic setting is that the entire concept of audience is, 

in a sense, fictional, as argued by Ong (1975). At least with professional writing for a mass 
audience, the writer is constructing a model of those likely to read his or her text and, by that 
act, constructing the text to appeal to a certain subset of the mass-market readership. The 
sophistication of inference and need to abstract over social context required for this kind of 
writing are, needless to say, considerably beyond that needed in a face-to-face dialogue. 

5 It is perhaps worthwhile in this context to note Graffs (2003) critique of academic discourse, 

which suggested that the multiplication of specialized terminologies, styles, and formulaic 
organization patterns can obscure the actual intellectual content in ways that can block 
comprehension of ideas that easily could be stated in more accessible ways. 

6 Some of the complexities involved are those specifically targeted in so-called postprocess 

theory (cfi, Kent, 2003). Many of the emphases of postprocess theorists are closely related to 
points raised in this review, including the public, socially situated, and interpreted nature of 
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writing. These considerations, however, do not obviate the need for close consideration of the 
cognitive processes necessary for writers to function effectively within their larger social and 
discourse contexts. 

7 We use the tenn strand intentionally to evoke the idea of interwoven elements that join together 
to form a more complex integrated skill. Similar metaphors may be found in Rutz and Lauer- 
Glebov (2005) and in Scarborough (2001). 
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